Merge branch '22348-gitea-importer' into 'master'

It adds a brand new importer for Gitea!

This is a continuation of !6945 started by @bkc.

Gitea aims to be 100% GitHub-compatible but there's a few differences:

- Gitea is not an OAuth provider (yet): https://github.com/go-gitea/gitea/issues/27
  - This means we cannot map Gitea users given an assignee ID => assignees are not set on imported issues and merge requests
- No releases API for now: https://github.com/go-gitea/gitea/issues/330
- API version is `v1` (GitHub is `v3`)
- The IID field for milestones is `id` compared to `number` in GitHub.
- Issues, PRs, milestones, labels don't have a `url` field (the importer now fallback to `''` in that case)

**Known issues:**

- Comments are not imported because comments JSON always have a blank `html_url`/`issue_url`/`pull_request_url`, so the IID cannot be extracted and the issuable cannot be found... :( This is tracked in https://github.com/go-gitea/gitea/issues/401, and solved by https://github.com/gogits/gogs/pull/3624 but this needs to be submitted / merged in Gitea.

This is noted in the documentation.

## Are there points in the code the reviewer needs to double check?

1. I've made `Import::GiteaController` inherit from `Import::GithubController` since both controllers should be identical in the long-term and their current differences are small.
1. I've added a base `IssuableFormatter` class from which `IssueFormatter` & `PullRequestFormatter` inherit
1. I've added shared examples for GitHub/Gitea importer classes
1. I've made `Gitlab::ImportSources` more robust and tested! :christmas_tree: 
1. I've added routing specs for import routes! :christmas_tree: 

Closes #22348

See merge request !8116

3 files changed, 111 insertions, 35 deletions

diff --git a/app/controllers/application_controller.rb b/app/controllers/application_controller.rb
index 4df80195ae1..bb47e2a8bf7 100644
--- a/app/controllers/application_controller.rb
+++ b/app/controllers/application_controller.rb
@@ -25,7 +25,7 @@ class ApplicationController < ActionController::Base
   protect_from_forgery with: :exception
 
   helper_method :can?, :current_application_settings
-  helper_method :import_sources_enabled?, :github_import_enabled?, :github_import_configured?, :gitlab_import_enabled?, :gitlab_import_configured?, :bitbucket_import_enabled?, :bitbucket_import_configured?, :google_code_import_enabled?, :fogbugz_import_enabled?, :git_import_enabled?, :gitlab_project_import_enabled?
+  helper_method :import_sources_enabled?, :github_import_enabled?, :gitea_import_enabled?, :github_import_configured?, :gitlab_import_enabled?, :gitlab_import_configured?, :bitbucket_import_enabled?, :bitbucket_import_configured?, :google_code_import_enabled?, :fogbugz_import_enabled?, :git_import_enabled?, :gitlab_project_import_enabled?
 
   rescue_from Encoding::CompatibilityError do |exception|
     log_exception(exception)
@@ -245,6 +245,10 @@ class ApplicationController < ActionController::Base
     current_application_settings.import_sources.include?('github')
   end
 
+  def gitea_import_enabled?
+    current_application_settings.import_sources.include?('gitea')
+  end
+
   def github_import_configured?
     Gitlab::OAuth::Provider.enabled?(:github)
   end
diff --git a/app/controllers/import/gitea_controller.rb b/app/controllers/import/gitea_controller.rb
new file mode 100644
index 00000000000..fbd851c64a7
--- /dev/null
+++ b/app/controllers/import/gitea_controller.rb
@@ -0,0 +1,45 @@
+class Import::GiteaController < Import::GithubController
+  def new
+    if session[access_token_key].present? && session[host_key].present?
+      redirect_to status_import_url
+    end
+  end
+
+  def personal_access_token
+    session[host_key] = params[host_key]
+    super
+  end
+
+  def status
+    @gitea_host_url = session[host_key]
+    super
+  end
+
+  private
+
+  def host_key
+    :"#{provider}_host_url"
+  end
+
+  # Overriden methods
+  def provider
+    :gitea
+  end
+
+  # Gitea is not yet an OAuth provider
+  # See https://github.com/go-gitea/gitea/issues/27
+  def logged_in_with_provider?
+    false
+  end
+
+  def provider_auth
+    if session[access_token_key].blank? || session[host_key].blank?
+      redirect_to new_import_gitea_url,
+        alert: 'You need to specify both an Access Token and a Host URL.'
+    end
+  end
+
+  def client_options
+    { host: session[host_key], api_version: 'v1' }
+  end
+end
diff --git a/app/controllers/import/github_controller.rb b/app/controllers/import/github_controller.rb
index ee7d498c59c..53a5981e564 100644
--- a/app/controllers/import/github_controller.rb
+++ b/app/controllers/import/github_controller.rb
@@ -1,39 +1,37 @@
 class Import::GithubController < Import::BaseController
-  before_action :verify_github_import_enabled
-  before_action :github_auth, only: [:status, :jobs, :create]
+  before_action :verify_import_enabled
+  before_action :provider_auth, only: [:status, :jobs, :create]
 
-  rescue_from Octokit::Unauthorized, with: :github_unauthorized
-
-  helper_method :logged_in_with_github?
+  rescue_from Octokit::Unauthorized, with: :provider_unauthorized
 
   def new
-    if logged_in_with_github?
-      go_to_github_for_permissions
-    elsif session[:github_access_token]
-      redirect_to status_import_github_url
+    if logged_in_with_provider?
+      go_to_provider_for_permissions
+    elsif session[access_token_key]
+      redirect_to status_import_url
     end
   end
 
   def callback
-    session[:github_access_token] = client.get_token(params[:code])
-    redirect_to status_import_github_url
+    session[access_token_key] = client.get_token(params[:code])
+    redirect_to status_import_url
   end
 
   def personal_access_token
-    session[:github_access_token] = params[:personal_access_token]
-    redirect_to status_import_github_url
+    session[access_token_key] = params[:personal_access_token]
+    redirect_to status_import_url
   end
 
   def status
     @repos = client.repos
-    @already_added_projects = current_user.created_projects.where(import_type: "github")
+    @already_added_projects = current_user.created_projects.where(import_type: provider)
     already_added_projects_names = @already_added_projects.pluck(:import_source)
 
-    @repos.reject!{ |repo| already_added_projects_names.include? repo.full_name }
+    @repos.reject! { |repo| already_added_projects_names.include? repo.full_name }
   end
 
   def jobs
-    jobs = current_user.created_projects.where(import_type: "github").to_json(only: [:id, :import_status])
+    jobs = current_user.created_projects.where(import_type: provider).to_json(only: [:id, :import_status])
     render json: jobs
   end
 
@@ -44,8 +42,8 @@ class Import::GithubController < Import::BaseController
     namespace_path = params[:target_namespace].presence || current_user.namespace_path
     @target_namespace = find_or_create_namespace(namespace_path, current_user.namespace_path)
 
-    if current_user.can?(:create_projects, @target_namespace)
-      @project = Gitlab::GithubImport::ProjectCreator.new(repo, @project_name, @target_namespace, current_user, access_params).execute
+    if can?(current_user, :create_projects, @target_namespace)
+      @project = Gitlab::GithubImport::ProjectCreator.new(repo, @project_name, @target_namespace, current_user, access_params, type: provider).execute
     else
       render 'unauthorized'
     end
@@ -54,34 +52,63 @@ class Import::GithubController < Import::BaseController
   private
 
   def client
-    @client ||= Gitlab::GithubImport::Client.new(session[:github_access_token])
+    @client ||= Gitlab::GithubImport::Client.new(session[access_token_key], client_options)
   end
 
-  def verify_github_import_enabled
-    render_404 unless github_import_enabled?
+  def verify_import_enabled
+    render_404 unless import_enabled?
   end
 
-  def github_auth
-    if session[:github_access_token].blank?
-      go_to_github_for_permissions
-    end
+  def go_to_provider_for_permissions
+    redirect_to client.authorize_url(callback_import_url)
   end
 
-  def go_to_github_for_permissions
-    redirect_to client.authorize_url(callback_import_github_url)
+  def import_enabled?
+    __send__("#{provider}_import_enabled?")
   end
 
-  def github_unauthorized
-    session[:github_access_token] = nil
-    redirect_to new_import_github_url,
-      alert: 'Access denied to your GitHub account.'
+  def new_import_url
+    public_send("new_import_#{provider}_url")
   end
 
-  def logged_in_with_github?
-    current_user.identities.exists?(provider: 'github')
+  def status_import_url
+    public_send("status_import_#{provider}_url")
+  end
+
+  def callback_import_url
+    public_send("callback_import_#{provider}_url")
+  end
+
+  def provider_unauthorized
+    session[access_token_key] = nil
+    redirect_to new_import_url,
+      alert: "Access denied to your #{Gitlab::ImportSources.title(provider.to_s)} account."
+  end
+
+  def access_token_key
+    :"#{provider}_access_token"
   end
 
   def access_params
-    { github_access_token: session[:github_access_token] }
+    { github_access_token: session[access_token_key] }
+  end
+
+  # The following methods are overriden in subclasses
+  def provider
+    :github
+  end
+
+  def logged_in_with_provider?
+    current_user.identities.exists?(provider: provider)
+  end
+
+  def provider_auth
+    if session[access_token_key].blank?
+      go_to_provider_for_permissions
+    end
+  end
+
+  def client_options
+    {}
   end
 end