Welcome to mirror list, hosted at ThFree Co, Russian Federation.

github.com/diaspora/diaspora.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
path: root/lib
diff options
context:
space:
mode:
authorcmrd Senya <senya@riseup.net>2019-04-26 18:41:27 +0300
committercmrd Senya <senya@riseup.net>2019-04-26 18:41:27 +0300
commitf85f167f505985c6c34dc77f5c29493e28d7aa20 (patch)
treeb11a3dd1f4ce569a7b485e485196463d65b955ee /lib
parente3c05b56200ff6ee83c6a4728417132262a7069f (diff)
Implement archive import backend
This implements archive import feature. The feature is divided in two main subfeatures: archive validation and archive import. Archive validation performs different validation on input user archive. This can be used without actually running import, e.g. when user wants to check the archive before import from the frontend. Validators may add messages and modify the archive. Validators are separated in two types: critical validators and non-critical validators. If validations by critical validators fail it means we can't import archive. If non-critical validations fail, we can import archive, but some warning messages are rendered. Also validators may change archive contents, e.g. when some entity can't be imported it may be removed from the archive. Validators' job is to take away complexity from the importer and perform the validations which are not implemented in other parts of the system, e.g. DB validations or diaspora_federation entity validations. Archive importer then takes the modified archive from the validator and imports it. In order to incapsulate high-level migration logic a MigrationService is introduced. MigrationService links ArchiveValidator, ArchiveImporter and AccountMigration. Also here is introduced a rake task which may be used by podmins to run archive import.
Diffstat (limited to 'lib')
-rw-r--r--lib/archive_importer.rb117
-rw-r--r--lib/archive_importer/archive_helper.rb45
-rw-r--r--lib/archive_importer/contact_importer.rb40
-rw-r--r--lib/archive_importer/entity_importer.rb30
-rw-r--r--lib/archive_importer/own_entity_importer.rb31
-rw-r--r--lib/archive_importer/own_relayable_importer.rb25
-rw-r--r--lib/archive_importer/post_importer.rb35
-rw-r--r--lib/archive_validator.rb60
-rw-r--r--lib/archive_validator/author_private_key_validator.rb17
-rw-r--r--lib/archive_validator/base_validator.rb27
-rw-r--r--lib/archive_validator/collection_validator.rb16
-rw-r--r--lib/archive_validator/contact_validator.rb41
-rw-r--r--lib/archive_validator/contacts_validator.rb13
-rw-r--r--lib/archive_validator/entities_helper.rb35
-rw-r--r--lib/archive_validator/others_relayables_validator.rb13
-rw-r--r--lib/archive_validator/own_relayable_validator.rb19
-rw-r--r--lib/archive_validator/post_validator.rb22
-rw-r--r--lib/archive_validator/posts_validator.rb13
-rw-r--r--lib/archive_validator/relayable_validator.rb66
-rw-r--r--lib/archive_validator/relayables_validator.rb13
-rw-r--r--lib/archive_validator/schema_validator.rb13
-rw-r--r--lib/diaspora/federated/fetchable.rb21
-rw-r--r--lib/tasks/accounts.rake44
23 files changed, 756 insertions, 0 deletions
diff --git a/lib/archive_importer.rb b/lib/archive_importer.rb
new file mode 100644
index 000000000..937f12480
--- /dev/null
+++ b/lib/archive_importer.rb
@@ -0,0 +1,117 @@
+# frozen_string_literal: true
+
+class ArchiveImporter
+ include ArchiveHelper
+ include Diaspora::Logging
+
+ attr_accessor :user
+
+ def initialize(archive_hash)
+ @archive_hash = archive_hash
+ end
+
+ def import
+ import_tag_followings
+ import_aspects
+ import_contacts
+ import_posts
+ import_relayables
+ import_subscriptions
+ import_others_relayables
+ end
+
+ def create_user(attr)
+ allowed_keys = %w[
+ email strip_exif show_community_spotlight_in_stream language disable_mail auto_follow_back
+ ]
+ data = convert_keys(archive_hash["user"], allowed_keys)
+ data.merge!(
+ username: attr[:username],
+ password: attr[:password],
+ password_confirmation: attr[:password]
+ )
+ self.user = User.build(data)
+ user.save!
+ end
+
+ private
+
+ attr_reader :archive_hash
+
+ def import_contacts
+ import_collection(contacts, ContactImporter)
+ end
+
+ def set_auto_follow_back_aspect
+ name = archive_hash["user"]["auto_follow_back_aspect"]
+ return if name.nil?
+
+ aspect = user.aspects.find_by(name: name)
+ user.update(auto_follow_back_aspect: aspect) if aspect
+ end
+
+ def import_aspects
+ contact_groups.each do |group|
+ begin
+ user.aspects.create!(group.slice("name", "chat_enabled"))
+ rescue ActiveRecord::RecordInvalid => e
+ logger.warn "#{self}: #{e}"
+ end
+ end
+ set_auto_follow_back_aspect
+ end
+
+ def import_posts
+ import_collection(posts, PostImporter)
+ end
+
+ def import_relayables
+ import_collection(relayables, OwnRelayableImporter)
+ end
+
+ def import_others_relayables
+ import_collection(others_relayables, EntityImporter)
+ end
+
+ def import_collection(collection, importer_class)
+ collection.each do |object|
+ importer_class.new(object, user).import
+ end
+ end
+
+ def import_tag_followings
+ archive_hash.fetch("user").fetch("followed_tags", []).each do |tag_name|
+ begin
+ tag = ActsAsTaggableOn::Tag.find_or_create_by(name: tag_name)
+ user.tag_followings.create!(tag: tag)
+ rescue ActiveRecord::RecordInvalid => e
+ logger.warn "#{self}: #{e}"
+ end
+ end
+ end
+
+ def import_subscriptions
+ post_subscriptions.each do |post_guid|
+ post = Post.find_or_fetch_by(archive_author_diaspora_id, post_guid)
+ if post.nil?
+ logger.warn "#{self}: post with guid #{post_guid} not found, can't subscribe"
+ next
+ end
+ begin
+ user.participations.create!(target: post)
+ rescue ActiveRecord::RecordInvalid => e
+ logger.warn "#{self}: #{e}"
+ end
+ end
+ end
+
+ def convert_keys(hash, allowed_keys)
+ hash
+ .slice(*allowed_keys)
+ .symbolize_keys
+ end
+
+ def to_s
+ "#{self.class}:#{archive_author_diaspora_id}:#{user.diaspora_handle}"
+ end
+end
diff --git a/lib/archive_importer/archive_helper.rb b/lib/archive_importer/archive_helper.rb
new file mode 100644
index 000000000..f0f8e7636
--- /dev/null
+++ b/lib/archive_importer/archive_helper.rb
@@ -0,0 +1,45 @@
+# frozen_string_literal: true
+
+class ArchiveImporter
+ module ArchiveHelper
+ def posts
+ @posts ||= archive_hash.fetch("user").fetch("posts", [])
+ end
+
+ def relayables
+ @relayables ||= archive_hash.fetch("user").fetch("relayables", [])
+ end
+
+ def others_relayables
+ @others_relayables ||= archive_hash.fetch("others_data", {}).fetch("relayables", [])
+ end
+
+ def post_subscriptions
+ archive_hash.fetch("user").fetch("post_subscriptions", [])
+ end
+
+ def contacts
+ archive_hash.fetch("user").fetch("contacts", [])
+ end
+
+ def contact_groups
+ @contact_groups ||= archive_hash.fetch("user").fetch("contact_groups", [])
+ end
+
+ def archive_author_diaspora_id
+ @archive_author_diaspora_id ||= archive_hash.fetch("user").fetch("profile").fetch("entity_data").fetch("author")
+ end
+
+ def person
+ @person ||= Person.find_or_fetch_by_identifier(archive_author_diaspora_id)
+ end
+
+ def private_key
+ OpenSSL::PKey::RSA.new(serialized_private_key)
+ end
+
+ def serialized_private_key
+ archive_hash.fetch("user").fetch("private_key")
+ end
+ end
+end
diff --git a/lib/archive_importer/contact_importer.rb b/lib/archive_importer/contact_importer.rb
new file mode 100644
index 000000000..492375691
--- /dev/null
+++ b/lib/archive_importer/contact_importer.rb
@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+class ArchiveImporter
+ class ContactImporter
+ include Diaspora::Logging
+
+ def initialize(json, user)
+ @json = json
+ @user = user
+ end
+
+ attr_reader :json
+ attr_reader :user
+
+ def import
+ @imported_contact = create_contact
+ add_to_aspects
+ rescue ActiveRecord::RecordInvalid => e
+ logger.warn "#{self}: #{e}"
+ end
+
+ private
+
+ def add_to_aspects
+ json.fetch("contact_groups_membership", []).each do |group_name|
+ aspect = user.aspects.find_by(name: group_name)
+ if aspect.nil?
+ logger.warn "#{self}: aspect \"#{group_name}\" is missing"
+ next
+ end
+ @imported_contact.aspects << aspect
+ end
+ end
+
+ def create_contact
+ person = Person.by_account_identifier(json.fetch("account_id"))
+ user.contacts.create!(person_id: person.id, sharing: false, receiving: json.fetch("receiving"))
+ end
+ end
+end
diff --git a/lib/archive_importer/entity_importer.rb b/lib/archive_importer/entity_importer.rb
new file mode 100644
index 000000000..6314c9e21
--- /dev/null
+++ b/lib/archive_importer/entity_importer.rb
@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+class ArchiveImporter
+ class EntityImporter
+ include ArchiveValidator::EntitiesHelper
+ include Diaspora::Logging
+
+ def initialize(json, user)
+ @json = json
+ @user = user
+ end
+
+ def import
+ self.persisted_object = Diaspora::Federation::Receive.perform(entity)
+ rescue DiasporaFederation::Entities::Signable::SignatureVerificationFailed,
+ DiasporaFederation::Discovery::InvalidDocument,
+ DiasporaFederation::Discovery::DiscoveryError,
+ ActiveRecord::RecordInvalid => e
+ logger.warn "#{self}: #{e}"
+ end
+
+ attr_reader :json
+ attr_reader :user
+ attr_accessor :persisted_object
+
+ def entity
+ entity_class.from_json(json)
+ end
+ end
+end
diff --git a/lib/archive_importer/own_entity_importer.rb b/lib/archive_importer/own_entity_importer.rb
new file mode 100644
index 000000000..2d585c0cd
--- /dev/null
+++ b/lib/archive_importer/own_entity_importer.rb
@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+class ArchiveImporter
+ class OwnEntityImporter < EntityImporter
+ def import
+ substitute_author
+ super
+ rescue Diaspora::Federation::InvalidAuthor
+ return if real_author == old_author_id
+
+ logger.warn "#{self.class}: attempt to import an entity with guid \"#{guid}\" which belongs to #{real_author}"
+ end
+
+ private
+
+ def substitute_author
+ @old_author_id = entity_data["author"]
+ entity_data["author"] = user.diaspora_handle
+ end
+
+ attr_reader :old_author_id
+
+ def persisted_object
+ @persisted_object ||= (instance if real_author == old_author_id)
+ end
+
+ def real_author
+ instance.author.diaspora_handle
+ end
+ end
+end
diff --git a/lib/archive_importer/own_relayable_importer.rb b/lib/archive_importer/own_relayable_importer.rb
new file mode 100644
index 000000000..a3325735a
--- /dev/null
+++ b/lib/archive_importer/own_relayable_importer.rb
@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+
+class ArchiveImporter
+ class OwnRelayableImporter < OwnEntityImporter
+ def entity
+ fetch_parent(symbolized_entity_data)
+ entity_class.new(symbolized_entity_data)
+ end
+
+ private
+
+ def symbolized_entity_data
+ @symbolized_entity_data ||= entity_data.slice(*entity_class.class_props.keys.map(&:to_s)).symbolize_keys
+ end
+
+ # Copied over from DiasporaFederation::Entities::Relayable
+ def fetch_parent(data)
+ type = data.fetch(:parent_type) {
+ break entity_class::PARENT_TYPE if entity_class.const_defined?(:PARENT_TYPE)
+ }
+ entity = Diaspora::Federation::Mappings.model_class_for(type).find_by(guid: data.fetch(:parent_guid))
+ data[:parent] = Diaspora::Federation::Entities.related_entity(entity)
+ end
+ end
+end
diff --git a/lib/archive_importer/post_importer.rb b/lib/archive_importer/post_importer.rb
new file mode 100644
index 000000000..79d592521
--- /dev/null
+++ b/lib/archive_importer/post_importer.rb
@@ -0,0 +1,35 @@
+# frozen_string_literal: true
+
+class ArchiveImporter
+ class PostImporter < OwnEntityImporter
+ include Diaspora::Logging
+
+ def import
+ super
+ import_subscriptions if persisted_object
+ end
+
+ private
+
+ def substitute_author
+ super
+ return unless entity_type == "status_message"
+
+ entity_data["photos"].each do |photo|
+ photo["entity_data"]["author"] = user.diaspora_handle
+ end
+ end
+
+ def import_subscriptions
+ json.fetch("subscribed_users_ids", []).each do |diaspora_id|
+ begin
+ person = Person.find_or_fetch_by_identifier(diaspora_id)
+ person = person.account_migration.newest_person unless person.account_migration.nil?
+ next if person.closed_account?
+ # TODO: unless person.nil? import subscription: subscription import is not supported yet
+ rescue DiasporaFederation::Discovery::DiscoveryError
+ end
+ end
+ end
+ end
+end
diff --git a/lib/archive_validator.rb b/lib/archive_validator.rb
new file mode 100644
index 000000000..4dda3df2e
--- /dev/null
+++ b/lib/archive_validator.rb
@@ -0,0 +1,60 @@
+# frozen_string_literal: true
+
+require "yajl"
+
+# ArchiveValidator checks for errors in archive. It also find non-critical problems and fixes them in the archive hash
+# so that the ArchiveImporter doesn't have to handle this issues. Non-critical problems found are indicated as warnings.
+# Also it performs necessary data fetch where required.
+class ArchiveValidator
+ include ArchiveImporter::ArchiveHelper
+
+ def initialize(archive)
+ @archive = archive
+ end
+
+ def validate
+ run_validators(CRITICAL_VALIDATORS, errors)
+ run_validators(NON_CRITICAL_VALIDATORS, warnings)
+ rescue KeyError => e
+ errors.push("Missing mandatory data: #{e}")
+ rescue Yajl::ParseError => e
+ errors.push("Bad JSON provided: #{e}")
+ end
+
+ def errors
+ @errors ||= []
+ end
+
+ def warnings
+ @warnings ||= []
+ end
+
+ def archive_hash
+ @archive_hash ||= Yajl::Parser.new.parse(archive)
+ end
+
+ CRITICAL_VALIDATORS = [
+ SchemaValidator,
+ AuthorPrivateKeyValidator
+ ].freeze
+
+ NON_CRITICAL_VALIDATORS = [
+ ContactsValidator,
+ PostsValidator,
+ RelayablesValidator,
+ OthersRelayablesValidator
+ ].freeze
+
+ private_constant :CRITICAL_VALIDATORS, :NON_CRITICAL_VALIDATORS
+
+ private
+
+ attr_reader :archive
+
+ def run_validators(list, messages)
+ list.each do |validator_class|
+ validator = validator_class.new(archive_hash)
+ messages.concat(validator.messages)
+ end
+ end
+end
diff --git a/lib/archive_validator/author_private_key_validator.rb b/lib/archive_validator/author_private_key_validator.rb
new file mode 100644
index 000000000..5d72d6f16
--- /dev/null
+++ b/lib/archive_validator/author_private_key_validator.rb
@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+class ArchiveValidator
+ class AuthorPrivateKeyValidator < BaseValidator
+ include Diaspora::Logging
+
+ def validate
+ return if person.nil?
+ return if person.serialized_public_key == private_key.public_key.export
+
+ messages.push("Private key in the archive doesn't match the known key of #{person.diaspora_handle}")
+ rescue DiasporaFederation::Discovery::DiscoveryError
+ logger.info "#{self}: Archive author couldn't be fetched (old home pod is down?), will continue with data"\
+ " import only"
+ end
+ end
+end
diff --git a/lib/archive_validator/base_validator.rb b/lib/archive_validator/base_validator.rb
new file mode 100644
index 000000000..183cafa22
--- /dev/null
+++ b/lib/archive_validator/base_validator.rb
@@ -0,0 +1,27 @@
+# frozen_string_literal: true
+
+class ArchiveValidator
+ class BaseValidator
+ include ArchiveImporter::ArchiveHelper
+ attr_reader :archive_hash
+
+ def initialize(archive_hash)
+ @archive_hash = archive_hash
+ validate
+ end
+
+ def messages
+ @messages ||= []
+ end
+
+ def valid?
+ @valid.nil? ? messages.empty? : @valid
+ end
+
+ private
+
+ attr_writer :valid
+
+ def validate; end
+ end
+end
diff --git a/lib/archive_validator/collection_validator.rb b/lib/archive_validator/collection_validator.rb
new file mode 100644
index 000000000..e03c2fe2f
--- /dev/null
+++ b/lib/archive_validator/collection_validator.rb
@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+class ArchiveValidator
+ class CollectionValidator < BaseValidator
+ # Runs validations over each element in collection and removes every element
+ # which fails the validations. Any messages produced by the entity_validator are
+ # concatenated to the messages of the CollectionValidator instance.
+ def validate
+ collection.keep_if do |item|
+ subvalidator = entity_validator.new(archive_hash, item)
+ messages.concat(subvalidator.messages)
+ subvalidator.valid?
+ end
+ end
+ end
+end
diff --git a/lib/archive_validator/contact_validator.rb b/lib/archive_validator/contact_validator.rb
new file mode 100644
index 000000000..4b79b0d30
--- /dev/null
+++ b/lib/archive_validator/contact_validator.rb
@@ -0,0 +1,41 @@
+# frozen_string_literal: true
+
+class ArchiveValidator
+ class ContactValidator < BaseValidator
+ def initialize(archive_hash, contact)
+ @contact = contact
+ super(archive_hash)
+ end
+
+ private
+
+ def validate
+ handle_migrant_contact
+ self.valid = account_open?
+ rescue DiasporaFederation::Discovery::DiscoveryError => e
+ messages.push("#{self.class}: failed to fetch person #{diaspora_id}: #{e}")
+ self.valid = false
+ end
+
+ attr_reader :contact
+
+ def diaspora_id
+ contact.fetch("account_id")
+ end
+
+ def handle_migrant_contact
+ return if person.account_migration.nil?
+
+ contact["account_id"] = person.account_migration.newest_person.diaspora_handle
+ @person = nil
+ end
+
+ def person
+ @person ||= Person.find_or_fetch_by_identifier(diaspora_id)
+ end
+
+ def account_open?
+ !person.closed_account? || (messages.push("#{self.class}: account #{diaspora_id} is closed") && false)
+ end
+ end
+end
diff --git a/lib/archive_validator/contacts_validator.rb b/lib/archive_validator/contacts_validator.rb
new file mode 100644
index 000000000..f7968012d
--- /dev/null
+++ b/lib/archive_validator/contacts_validator.rb
@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+class ArchiveValidator
+ class ContactsValidator < CollectionValidator
+ def collection
+ contacts
+ end
+
+ def entity_validator
+ ContactValidator
+ end
+ end
+end
diff --git a/lib/archive_validator/entities_helper.rb b/lib/archive_validator/entities_helper.rb
new file mode 100644
index 000000000..40c47a59a
--- /dev/null
+++ b/lib/archive_validator/entities_helper.rb
@@ -0,0 +1,35 @@
+# frozen_string_literal: true
+
+class ArchiveValidator
+ module EntitiesHelper
+ private
+
+ def instance
+ @instance ||= model_class.find_by(guid: guid)
+ end
+
+ def entity_type
+ json.fetch("entity_type")
+ end
+
+ def entity_data
+ json.fetch("entity_data")
+ end
+
+ def model_class
+ @model_class ||= Diaspora::Federation::Mappings.model_class_for(entity_type.camelize)
+ end
+
+ def entity_class
+ DiasporaFederation::Entity.entity_class(entity_type)
+ end
+
+ def guid
+ @guid ||= entity_data.fetch("guid")
+ end
+
+ def to_s
+ "#{entity_class.class_name}:#{guid}"
+ end
+ end
+end
diff --git a/lib/archive_validator/others_relayables_validator.rb b/lib/archive_validator/others_relayables_validator.rb
new file mode 100644
index 000000000..7352c9957
--- /dev/null
+++ b/lib/archive_validator/others_relayables_validator.rb
@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+class ArchiveValidator
+ class OthersRelayablesValidator < CollectionValidator
+ def collection
+ others_relayables
+ end
+
+ def entity_validator
+ RelayableValidator
+ end
+ end
+end
diff --git a/lib/archive_validator/own_relayable_validator.rb b/lib/archive_validator/own_relayable_validator.rb
new file mode 100644
index 000000000..01b8dfdbe
--- /dev/null
+++ b/lib/archive_validator/own_relayable_validator.rb
@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+class ArchiveValidator
+ class OwnRelayableValidator < RelayableValidator
+ private
+
+ def post_find_by_guid(guid)
+ super || by_guid(Post, guid)
+ end
+
+ def post_find_by_poll_guid(guid)
+ super || by_guid(Poll, guid)&.status_message
+ end
+
+ def by_guid(klass, guid)
+ klass.find_or_fetch_by(archive_author_diaspora_id, guid)
+ end
+ end
+end
diff --git a/lib/archive_validator/post_validator.rb b/lib/archive_validator/post_validator.rb
new file mode 100644
index 000000000..f1f2bf142
--- /dev/null
+++ b/lib/archive_validator/post_validator.rb
@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+class ArchiveValidator
+ class PostValidator < BaseValidator
+ include EntitiesHelper
+
+ def initialize(archive_hash, post)
+ @json = post
+ super(archive_hash)
+ end
+
+ private
+
+ def validate
+ return unless entity_type == "reshare" && entity_data["root_guid"].nil?
+
+ messages.push("reshare #{self} doesn't have a root, ignored")
+ end
+
+ attr_reader :json
+ end
+end
diff --git a/lib/archive_validator/posts_validator.rb b/lib/archive_validator/posts_validator.rb
new file mode 100644
index 000000000..fb2a1d744
--- /dev/null
+++ b/lib/archive_validator/posts_validator.rb
@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+class ArchiveValidator
+ class PostsValidator < CollectionValidator
+ def collection
+ posts
+ end
+
+ def entity_validator
+ PostValidator
+ end
+ end
+end
diff --git a/lib/archive_validator/relayable_validator.rb b/lib/archive_validator/relayable_validator.rb
new file mode 100644
index 000000000..733848c57
--- /dev/null
+++ b/lib/archive_validator/relayable_validator.rb
@@ -0,0 +1,66 @@
+# frozen_string_literal: true
+
+class ArchiveValidator
+ # We have to validate relayables before import because during import we'll not be able to fetch parent anymore
+ # because parent author will point to ourselves.
+ class RelayableValidator < BaseValidator
+ include EntitiesHelper
+
+ def initialize(archive_hash, relayable)
+ @relayable = relayable
+ super(archive_hash)
+ end
+
+ private
+
+ def validate
+ self.valid = parent_present?
+ end
+
+ attr_reader :relayable
+ alias json relayable
+
+ # TODO: use diaspora federation to fetch parent where possible
+ # For own relayables we could just use RelatedEntity.fetch;
+ # For others' relayables we should check the present "own posts" first, and then if the target post is missing from
+ # there we could try to fetch it with RelatedEntity.fetch.
+
+ # Common methods used by subclasses:
+
+ def missing_parent_message
+ messages.push("Parent entity for #{self} is missing. Impossible to import, ignoring.")
+ end
+
+ def parent_present?
+ parent.present? || (missing_parent_message && false)
+ end
+
+ def parent
+ @parent ||= find_parent
+ end
+
+ def find_parent
+ if entity_type == "poll_participation"
+ post_find_by_poll_guid(parent_guid)
+ else
+ post_find_by_guid(parent_guid)
+ end
+ end
+
+ def parent_guid
+ entity_data.fetch("parent_guid")
+ end
+
+ def post_find_by_guid(guid)
+ posts.find {|post|
+ post.fetch("entity_data").fetch("guid") == guid
+ }
+ end
+
+ def post_find_by_poll_guid(guid)
+ posts.find {|post|
+ post.fetch("entity_data").fetch("poll", nil)&.fetch("entity_data", nil)&.fetch("guid", nil) == guid
+ }
+ end
+ end
+end
diff --git a/lib/archive_validator/relayables_validator.rb b/lib/archive_validator/relayables_validator.rb
new file mode 100644
index 000000000..ecfdb6a04
--- /dev/null
+++ b/lib/archive_validator/relayables_validator.rb
@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+class ArchiveValidator
+ class RelayablesValidator < CollectionValidator
+ def collection
+ relayables
+ end
+
+ def entity_validator
+ OwnRelayableValidator
+ end
+ end
+end
diff --git a/lib/archive_validator/schema_validator.rb b/lib/archive_validator/schema_validator.rb
new file mode 100644
index 000000000..1d4d1ab64
--- /dev/null
+++ b/lib/archive_validator/schema_validator.rb
@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+class ArchiveValidator
+ class SchemaValidator < BaseValidator
+ JSON_SCHEMA = "lib/schemas/archive-format.json"
+
+ def validate
+ return if JSON::Validator.validate(JSON_SCHEMA, archive_hash)
+
+ messages.push("Archive schema validation failed")
+ end
+ end
+end
diff --git a/lib/diaspora/federated/fetchable.rb b/lib/diaspora/federated/fetchable.rb
new file mode 100644
index 000000000..b0ab24290
--- /dev/null
+++ b/lib/diaspora/federated/fetchable.rb
@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+module Diaspora
+ module Federated
+ module Fetchable
+ extend ActiveSupport::Concern
+
+ module ClassMethods
+ def find_or_fetch_by(diaspora_id, guid)
+ instance = find_by(guid: guid)
+ return instance if instance.present?
+
+ DiasporaFederation::Federation::Fetcher.fetch_public(diaspora_id, to_s, guid)
+ find_by(guid: guid)
+ rescue DiasporaFederation::Federation::Fetcher::NotFetchable
+ nil
+ end
+ end
+ end
+ end
+end
diff --git a/lib/tasks/accounts.rake b/lib/tasks/accounts.rake
new file mode 100644
index 000000000..f7a17adf3
--- /dev/null
+++ b/lib/tasks/accounts.rake
@@ -0,0 +1,44 @@
+# frozen_string_literal: true
+
+namespace :accounts do
+ desc "Perform migration"
+ task :migration, %i[archive_path new_user_name] => :environment do |_t, args|
+ puts "Account migration is requested"
+ args = %i[archive_path new_user_name].map {|name| [name, args[name]] }.to_h
+ process_arguments(args)
+
+ begin
+ service = MigrationService.new(args[:archive_path], args[:new_user_name])
+ service.validate
+ puts "Warnings:\n#{service.warnings}\n-----" if service.warnings.any?
+ if service.only_import?
+ puts "Warning: Archive owner is not fetchable. Proceeding with data import, but account migration record "\
+ "won't be created"
+ end
+ print "Do you really want to execute the archive import? Note: this is irreversible! [y/N]: "
+ next unless $stdin.gets.strip.casecmp?("y")
+
+ start_time = Time.now.getlocal
+ service.perform!
+ puts service.only_import? ? "Data import complete!" : "Data import and migration complete!"
+ puts "Migration took #{Time.now.getlocal - start_time} seconds"
+ rescue MigrationService::ArchiveValidationFailed => exception
+ puts "Errors in the archive found:\n#{exception.message}\n-----"
+ rescue MigrationService::MigrationAlreadyExists
+ puts "Migration record already exists for the user, can't continue"
+ end
+ end
+
+ def process_arguments(args)
+ if args[:archive_path].nil?
+ print "Enter the archive path: "
+ args[:archive_path] = $stdin.gets.strip
+ end
+ if args[:new_user_name].nil?
+ print "Enter the new user name: "
+ args[:new_user_name] = $stdin.gets.strip
+ end
+ puts "Archive path: #{args[:archive_path]}"
+ puts "New username: #{args[:new_user_name]}"
+ end
+end