diff options
author | João Cunha <j.a.cunha@gmail.com> | 2019-04-09 22:45:58 +0300 |
---|---|---|
committer | João Cunha <j.a.cunha@gmail.com> | 2019-05-29 13:21:53 +0300 |
commit | a2aa160cea36fd8969e38eafb352154ee7d8f6f0 (patch) | |
tree | 414156bf3b3bba4751e6287ebf8ddf6717f6b35f /app/finders | |
parent | 328740c61327f20c18b43b5015d4411897a24c3c (diff) |
Adapt functions to work for external Knative
Remove Kn services cache from Clusters::Application::Knative
Knative function can exist even if user did not installed Knative via
GitLab managed apps.
-> Move responsibility of finding services into the Cluster
-> Responsability is inside Clusters::Cluster::KnativeServiceFinder
-> Projects::Serverless::FunctionsFinder now calls depends solely on a
cluster to find the Kn services.
-> Detect Knative by resource presence instead of service presence
-> Mock knative_installed response temporarily for frontend to develop
Display loader while `installed === 'checking'`
Added frontend work to determine if Knative is installed
Memoize with_reactive_cache(*args, &block) to avoid race conditions
When calling with_reactive_cache more than once, it's possible that the
second call will already have the value populated. Therefore, in cases
where we need the sequential calls to have consistent results, we'd fall
under a race condition.
Check knative installation via Knative resource presence
Only load pods if Knative is discovered
Always return a response in FunctionsController#index
- Always indicate if Knative is installed, not installed or checking
- Always indicate the partial response for functions. Final response is
guaranteed when knative_installed is either true | false.
Adds specs for Clusters::Cluster#knative_services_finder
Fix method name when calling on specs
Add an explicit check for functions
Added an explicit check to see if there are any functions available
Fix Serverless feature spec
- we don't find knative installation via database anymore,
rather via Knative resource
Display error message for request timeouts
Display an error message if the request times out
Adds feature specs for when functions exist
Remove a test purposed hardcoded flag
Add ability to partially load functions
Added the ability to partially load functions on the frontend
Add frontend unit tests
Added tests for the new frontend additions
Generate new translations
Generated new frontend translations
Address review comments
Cleaned up the frontend unit test.
Added computed prop for `isInstalled`.
Move string to constant
Simplify nil to array conversion
Put knative_installed states in a frozen hash for better read
Pluralize list of Knative states
Quey services and pods filtering name
This way we don't need to filter the namespace in memory.
Also, the data we get from the network is much smaller.
Simplify cache_key and fix bug
- Simplifies the cache_key by removing namespace duplicate
- Fixes a bug with reactive_cache memoization
Diffstat (limited to 'app/finders')
-rw-r--r-- | app/finders/clusters/cluster/knative_services_finder.rb | 114 | ||||
-rw-r--r-- | app/finders/projects/serverless/functions_finder.rb | 39 |
2 files changed, 138 insertions, 15 deletions
diff --git a/app/finders/clusters/cluster/knative_services_finder.rb b/app/finders/clusters/cluster/knative_services_finder.rb new file mode 100644 index 00000000000..85b0967e935 --- /dev/null +++ b/app/finders/clusters/cluster/knative_services_finder.rb @@ -0,0 +1,114 @@ +# frozen_string_literal: true +module Clusters + class Cluster + class KnativeServicesFinder + include ReactiveCaching + include Gitlab::Utils::StrongMemoize + + KNATIVE_STATES = { + 'checking' => 'checking', + 'installed' => 'installed', + 'not_found' => 'not_found' + }.freeze + + self.reactive_cache_key = ->(finder) { finder.model_name } + self.reactive_cache_worker_finder = ->(_id, *cache_args) { from_cache(*cache_args) } + + attr_reader :cluster, :project + + def initialize(cluster, project) + @cluster = cluster + @project = project + end + + def with_reactive_cache_memoized(*cache_args, &block) + strong_memoize(:reactive_cache) do + with_reactive_cache(*cache_args, &block) + end + end + + def clear_cache! + clear_reactive_cache!(*cache_args) + end + + def self.from_cache(cluster_id, project_id) + cluster = Clusters::Cluster.find(cluster_id) + project = ::Project.find(project_id) + + new(cluster, project) + end + + def calculate_reactive_cache(*) + # read_services calls knative_client.discover implicitily. If we stop + # detecting services but still want to detect knative, we'll need to + # explicitily call: knative_client.discover + # + # We didn't create it separately to avoid 2 cluster requests. + ksvc = read_services + pods = knative_client.discovered ? read_pods : [] + { services: ksvc, pods: pods, knative_detected: knative_client.discovered } + end + + def services + return [] unless search_namespace + + cached_data = with_reactive_cache_memoized(*cache_args) { |data| data } + cached_data.to_h.fetch(:services, []) + end + + def cache_args + [cluster.id, project.id] + end + + def service_pod_details(service) + cached_data = with_reactive_cache_memoized(*cache_args) { |data| data } + cached_data.to_h.fetch(:pods, []).select do |pod| + filter_pods(pod, service) + end + end + + def knative_detected + cached_data = with_reactive_cache_memoized(*cache_args) { |data| data } + + knative_state = cached_data.to_h[:knative_detected] + + return KNATIVE_STATES['checking'] if knative_state.nil? + return KNATIVE_STATES['installed'] if knative_state + + KNATIVE_STATES['uninstalled'] + end + + def model_name + self.class.name.underscore.tr('/', '_') + end + + private + + def search_namespace + @search_namespace ||= cluster.kubernetes_namespace_for(project) + end + + def knative_client + cluster.kubeclient.knative_client + end + + def filter_pods(pod, service) + pod["metadata"]["labels"]["serving.knative.dev/service"] == service + end + + def read_services + knative_client.get_services(namespace: search_namespace).as_json + rescue Kubeclient::ResourceNotFoundError + [] + end + + def read_pods + cluster.kubeclient.core_client.get_pods(namespace: search_namespace).as_json + end + + def id + nil + end + end + end +end diff --git a/app/finders/projects/serverless/functions_finder.rb b/app/finders/projects/serverless/functions_finder.rb index e5bffccabfe..9b8d7ed5a58 100644 --- a/app/finders/projects/serverless/functions_finder.rb +++ b/app/finders/projects/serverless/functions_finder.rb @@ -14,8 +14,15 @@ module Projects knative_services.flatten.compact end - def installed? - clusters_with_knative_installed.exists? + # Possible return values: Clusters::Cluster::KnativeServicesFinder::KNATIVE_STATE + def knative_installed + states = @clusters.map do |cluster| + cluster.knative_services_finder(project).knative_detected.tap do |state| + return state if state == ::Clusters::Cluster::KnativeServicesFinder::KNATIVE_STATES['checking'] # rubocop:disable Cop/AvoidReturnFromBlocks + end + end + + states.any? { |state| state == ::Clusters::Cluster::KnativeServicesFinder::KNATIVE_STATES['installed'] } end def service(environment_scope, name) @@ -25,7 +32,7 @@ module Projects def invocation_metrics(environment_scope, name) return unless prometheus_adapter&.can_query? - cluster = clusters_with_knative_installed.preload_knative.find do |c| + cluster = @clusters.find do |c| environment_scope == c.environment_scope end @@ -34,7 +41,7 @@ module Projects end def has_prometheus?(environment_scope) - clusters_with_knative_installed.preload_knative.to_a.any? do |cluster| + @clusters.any? do |cluster| environment_scope == cluster.environment_scope && cluster.application_prometheus_available? end end @@ -42,10 +49,12 @@ module Projects private def knative_service(environment_scope, name) - clusters_with_knative_installed.preload_knative.map do |cluster| + @clusters.map do |cluster| next if environment_scope != cluster.environment_scope - services = cluster.application_knative.services_for(ns: cluster.kubernetes_namespace_for(project)) + services = cluster + .knative_services_finder(project) + .services .select { |svc| svc["metadata"]["name"] == name } add_metadata(cluster, services).first unless services.nil? @@ -53,8 +62,11 @@ module Projects end def knative_services - clusters_with_knative_installed.preload_knative.map do |cluster| - services = cluster.application_knative.services_for(ns: cluster.kubernetes_namespace_for(project)) + @clusters.map do |cluster| + services = cluster + .knative_services_finder(project) + .services + add_metadata(cluster, services) unless services.nil? end end @@ -65,17 +77,14 @@ module Projects s["cluster_id"] = cluster.id if services.length == 1 - s["podcount"] = cluster.application_knative.service_pod_details( - cluster.kubernetes_namespace_for(project), - s["metadata"]["name"]).length + s["podcount"] = cluster + .knative_services_finder(project) + .service_pod_details(s["metadata"]["name"]) + .length end end end - def clusters_with_knative_installed - @clusters.with_knative_installed - end - # rubocop: disable CodeReuse/ServiceClass def prometheus_adapter @prometheus_adapter ||= ::Prometheus::AdapterService.new(project).prometheus_adapter |