Enable Discovery Handlers as Pods #252

kate-goldenring · 2021-02-23T21:34:34Z

What this PR does / why we need it:
This PR enables Discovery Handlers to run outside of the Akri Agent (closes #198 )!! To users, this PR currently does not change Akri usage, as the Agent is still being built with all discovery handlers embedded. However, new Discovery Handler pods that implement Akri's gRPC discovery interface can now be deployed and utilized.

The next iteration of this PR will enable a slim Agent build and include user guide updates to describe how to leverage the slim Agent. The Helm templates already include Akri's supported discovery handlers as Daemonsets and the values needed to deploy them with a locally built slim Agent. Just set the feature agent-modular in the Agent's Cargo.toml to build it without embedded discovery handlers.

A template for kick-starting discovery handler development was created and currently hosted in my personal repository. It is referenced from extensibility.md.

This is a breaking change to Akri, as it changed the Configuration CRD to not be closely tied to certain supported protocols.

Special notes for your reviewer:
Instead of crowding this description, I created a document in our HackMD that details everything that is included in the PR and should help reviewers navigate the HUGE amount of changes. Feel free to leave any questions/comments on the doc.

Testing the DiscoveryOperator was a headache and resulted in a bit of a mess. I created a document to explain the headaches and the pieced together (hopefully only temporary) solution.

For more information on the how the design was chosen, see the design document.

Also closes #133, closes #104, ref #145 (mounts udev devices via DeviceSpec but still need to enable passing permissions via Configuration)

If applicable:

this PR contains documentation
this PR contains unit tests
added code adheres to standard Rust formatting (cargo fmt)
code builds properly (cargo build)
code is free of common mistakes (cargo clippy)
all Akri tests succeed (cargo test)
inline documentation builds (cargo doc)
version has been updated appropriately (./version.sh)

.dockerignore

test/yaml/akri-opcua.yaml

test/yaml/akri-onvif-video.yaml

test/yaml/akri-debug-echo-foo.yaml

shared/src/lib.rs

agent/Cargo.toml

docs/extensibility.md

kate-goldenring · 2021-02-23T22:13:44Z

agent/src/util/device_plugin_builder.rs

+#[async_trait]
+impl DevicePluginBuilderInterface for DevicePluginBuilder {
+    /// This creates a new DevicePluginService for an instance and registers it with the kubelet
+    async fn build_device_plugin(


These methods have been moved from device_plugin_service.rs into a Trait so as to enable mocking and testing. They have not been modified.

kate-goldenring · 2021-02-23T22:18:29Z

agent/src/util/discovery_operator.rs

+                }
+            }
+        } else {
+            match DiscoveryClient::connect(endpoint.to_string()).await {


While discovery handlers always have to register with the Agents unix socket, the Agent supports connecting with the Discovery Handler over uds or IP. The use case for IP is narrow but could be helpful for the scenario where someone may want to run a Discovery Handler outside the cluster. They would need to deploy some sort of job to the cluster that would register the external Discovery Handler, specifying the IP endpoint during registration. Then the Agent would call the external one.

kate-goldenring · 2021-02-23T22:21:52Z

discovery-handler-modules/debug-echo-discovery-handler/src/main.rs

+    // Determine whether to serve discovery handler over UDS or IP based on existence
+    // of the environment variable POD_IP.
+    let mut use_uds = true;
+    let mut endpoint: String = match std::env::var("POD_IP") {


All of Akri's supported Discovery Handlers support running serving their discovery service over IP and UDS. Should the IP support be removed from the Discovery Handlers? It may just crowd it out and will probably not be used. It does provide an example for how one may configure their discovery handler to run over IP.

bfjelds · 2021-02-23T22:58:13Z

deployment/helm/templates/agent.yaml

@@ -77,6 +79,9 @@ spec:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      volumes:
+      - name: agent-registration
+        hostPath:
+          path: /var/lib/akri


this is an ignorance question, but is it strange to have a container create/own a path in /var/lib? are there any permission issues we could run into?

Hmm good point -- more investigation should happen here. I just mimicked device plugin because our agent is just a middle man of sorts to Device Plugin.

it doesn't seem entirely uncommon to use this path in a kubernetes application (grafana, zookeeper, cassandra, csi): https://github.com/search?p=2&q=%22%2Fvar%2Flib%22+extension%3Ayaml+mountpath+-virtlet+-heketi+-openvswitch&type=Code

kept this as default but made configurable in 00cc473

agent/src/main.rs

agent/src/util/discovery_operator.rs

romoh · 2021-02-23T22:57:16Z

agent/src/util/discovery_operator.rs

+        let config_name = config.metadata.name.clone();
+        let mut tasks = Vec::new();
+        let discovery_operator = Arc::new(discovery_operator);
+        let task1_discovery_operator = discovery_operator.clone();


I'm not clear on the difference between task1 and task2. It seems one is always only discovering while the other monitors for stopping as well?

I updated the documentation on start_discovery to delineate what each task does in 80e02b7. Task1 and Task2 are comparable to how agent and controller check for pre-existing and new configurations and instances, respectively. Here, start_discovery is checking for already registered and newly registered discovery handlers. The second task has a channel to notify it to stop checking when the associated Configuration is deleted.

agent/src/util/discovery_operator.rs

deployment/helm/templates/custom-discovery.yaml

Cargo.toml

deployment/helm/values.yaml

discovery-handler-modules/debug-echo-discovery-handler/src/main.rs

agent/src/util/discovery_operator.rs

.dockerignore

agent/src/util/discovery_operator.rs

docs/extensibility.md

discovery-handlers/onvif/src/lib.rs

agent/src/util/registration.rs

… tests

agent/src/util/constants.rs

jiria · 2021-03-09T21:02:59Z

shared/src/uds/unix_stream.rs

+    let mut connected = false;
+    let start = SystemTime::now()
+        .duration_since(UNIX_EPOCH)
+        .expect("Time went backwards")


Why Time went backwards comment?

An error only occurs here if sytime clock jumps. It seems to be common usage to say this happened with expect when using SystemTime than to unwrap which will display the amount of time the clock was unexpectedly off by.

jiria · 2021-03-09T21:06:37Z

shared/src/k8s/mod.rs

+                }
+            }
+        }
+        random_delay().await;


What is the cap on the random_delay?

up to 200 ms. It is noted in in-line doc of the function. should it also be noted elsewhere?

jiria · 2021-03-09T21:09:10Z

shared/src/k8s/mod.rs

+    instance_name: &str,
+    instance_namespace: &str,
+) -> Result<(), Box<dyn std::error::Error + Send + Sync + 'static>> {
+    for x in 0..MAX_INSTANCE_UPDATE_TRIES {


Do we really need to retry even for a delete operation?

I was thinking we needed to in case there is an error reaching api server. Do you expect this to not be the case for deletes?

…in tests

kate-goldenring · 2021-03-10T17:01:30Z

This is currently being blocked due to an issue running MicroK8s version 1.19 in GitHub actions: canonical/microk8s#2083

* test latest k8s versions [SAME VERSION] (#188) * update changelog for 0.1.5 release (#189) * add more description and simplify commands in e2e demo (#191) * Various typos fixed on new OPCUA documentation (#195) * Various typos fixed on new OPCUA documentation Signed-off-by: Didier Durand <[email protected]> * Fixes on extensibility documentation Signed-off-by: Didier Durand <[email protected]> * Make end-to-end demo on Raspberry Pi simpler and more descriptive (#193) * mirror updates in e2e demo in rasp-pi demo * syntax changes, capacity specification, building kernel module * fix syntax issues * Get Agent logs before testing their contents [SAME VERSION] (#197) * add additional checks to see why getting logs failing * log information when get crictl error * use os.system to log info * try using shell with subprocess when doing complex commands * Fix failing to query container runtime in test workflow on MicroK8s [SAME VERSION] (#202) * loop query containerd several times instead of sleeping * shorten agent log query * Allow overwriting controller.nodeSelectors (#194) * Allow overwriting controller.nodeSelectors * Add note about `sed` for MacOS contributors * Allow conditional compilation of agent protocols (#196) * Allow conditional compilation of agent protocols * Bump patch version Co-authored-by: Jiayi Hu <[email protected]> * Prometheus (#190) * add frame count metric to udev broker * add prometheus metrics * format metrics.rs * update Helm charts for Prometheus support * Prometheus documentation * increase supported rust version to latest * fix metrics chart * use PodMonitor instead of ServiceMonitor for Akri Agent and Controller * update tarpaulin to latest version to fix protobuf issue * remove debug echo shared setting * update version * Use anyhow instead of the deprecated failure crate (#201) * Switch to use anyhow crate instead of failure crate Include Cargo.lock updates * Rebase and increment patch version * Merge conflict * Add missing version updates after rebase * [Helm] Added CRD variables (#205) * Added CRD variables * `version.sh` updates Helm `crds.version` * Bump version * Document naming convention (#204) * Initial commit * Remove references to `http` example * Added `udev-broker` * Added `akri-udev` * Rephrased * Typo * naming-guidelines >> development.md * r/udev-broker/udev-video-broker * punctuation * r/Resources/resources * r/Compute/Computer (#210) * fix instructions for cloning and building v4l2 (#209) * Update hyper version and other dependencies * Update patch version * Explicitly specify h2 patch version dependency to avoid future accidentl updates by cargo-update or similar * Remove redundant unnecessary hyper dependencies * Add security audit workflow * Move security audit to existing check-rust workflow * Simplify mockall usage (#214) * Simplify mockall usage * Bump patch version * add project logo and art [SAME VERSION] (#149) * add akri branding and artwork Signed-off-by: flynnduism <[email protected]> * edit artwork README Signed-off-by: flynnduism <[email protected]> * re-render dark logos Signed-off-by: flynnduism <[email protected]> * redraw the svg icon to fix glitchy render Signed-off-by: flynnduism <[email protected]> * update logo.svg Signed-off-by: flynnduism <[email protected]> * fix the mixed up color hex values Signed-off-by: flynnduism <[email protected]> * update branding with inconsolata typeface Signed-off-by: flynnduism <[email protected]> * use correct casing for mentions of Akri Signed-off-by: flynnduism <[email protected]> * remove default documentation to use latest containers (#218) * Image pull policy (#207) * change default image pull policy to IfNotPresent * set ImagePullPolicy to default to empty * increase version * only add ImagePullPolicy if specified * Add a workflow to automated dependencies update * fix the instruction of the end to end demo and the rspi4 demo. (#217) * Add comment for cargo-audit action behavior * use akri bot PAT for the autoupdate workflow * update token secret name * Patch anomaly detection app (#229) * close connection to grpc service each loop * update version * use local subchannel pool * fix title typo * Add logo to README [SAME VERSION] (#220) * Auto-updated dependencies * Auto-updated dependencies * Allow unreachable code introduced with feature-driven protocols [SAME VERSION] (#232) * Allow unreachable code introduced with feature-driven protocols [SAME VERSION] The catchall match is there in case someone turns off one of the handlers via features ... but at runtime the disabled feature is configured. * update for cargo fmt * add comment explanation * fix cargo fmt * Improve development experience with timestamped container labels (#234) Using labels based on version.txt can make it difficult to iterate during development. Creating timestamped labels for the local build experience should make it easier to differentiate between development builds. * Reduce frequency of dependency update [SAME VERSION] (#236) * Reduce frequency of dependency update Our dependencies update on a daily basis ... not sure it makes sense for us to update our version each time someone else's do. I chose a monthly cadence because it would, at most, cause us to update our version 12x per year, which doesn't seem too bad. * Update auto-update-dependencies.yml * update code owners * Lazy eval of makefile timestamp breaks local build [SAME VERSION] [IGNORE INTERMEDIATE BUILDS] (#242) * Update hyper crate for security risk * patch version update * force override kubernetes package to deal with installation errors * Fix typo * Simplify matrix; Use all versions 1.16+ for all distros (#244) * Simplify matrix; Use all versions 1.16+ for all distros * Remove non-existent webhook test * Remove `microk8s.enable helm3` appears redundant and fails * Link Kubernetes and crictl for K3s|MicroK8s * Per: #206 (comment) * Typo * fix matrix and crictl path * Consistency and double-quote to expand `${PWD}` * Temporarily (!) block K3s 1.16 due to hard-coded device plugs (k3s-io/k3s#1390) * Test mapping K3s 1.16 device plugins to default K8s location * Create kubectl path in expected location if not present * Consistent w/ MicroK8s step * Wait until cluster has stabilized before wrangling device-plugins * Initial documentation for `run-test-cases.yml` * Correct indentation Co-authored-by: bfjelds <[email protected]> * add microk8s inspect to installation * try waiting repeatedly for microk8s to be ready * Cleanup streaming app sample (#173) * Initial version * Create logical classes * Fixes and optimizations * Use naming guideline for method names * Version bump * Version bump Co-authored-by: Brian Fjeldstad <[email protected]> * Admission Controller (Webhook) for Akri Configuration(s) (#206) * Helm Chart updates * Add 'webhooks' to the project * Commented Helm Chart `webhook-configuration` * Template `webhook-configuration` * Webhook service * Dockerfile (draft) * GitHub Action (draft) * Force build * Trying again * And again * Added GHCR secret to fork * Grrr * Grrr * Issue w/ GitHub Action build * Undo * Consistency * Revised Rust Webhook to not require these * Preliminary README * Update `version.sh` and run `./version.sh -u -s` * Added `webhook-configuration` * Does this resolve Helm Linting concern? * r/AMD32V7/AMR32V7 * Are these required? * Follow pattern * Follow Akri pattern * r/webhook-configuration/webhookConfiguration * correct path references * Cross-build Rust * Exceptions no longer required * Assume `./target/release/webhook-configuration` * Expose x-plat targets to docker * Omission * Improve handling of flags * Tidy * Correct indendation * Version bump * Update `CHANGELOG` * `cargo.lock` versions now bumped too * Temporarily (!) bumping build timeouts * Corrected component reference * Missed `build-arg=CROSS_BUILD_TARGET=...` * Avoid Helm issue with un-`named` `Lists` * Revert `CHANGELOG` * Added unit tests * Filters Akri Configuration manifest so that it validates * Added test for filtered `generation` * Added test for extended (complete) Config * To avoid GitHub Actions build failure * End-to-End Webhook tests * Integration tests require `actix-rt` * Include `webhook-configuration` container image * Learns all the things! * Typo * Added `webhook` to `test-case`s * Removed `get_agent_and_controller` reference * Simplified Actions `matrix`; Updated Webhook certs * Override `webhookConfiguration.image.tag` to `pr-amd64` * Add debugging * Typo * sudo'ing `kubectl` commands * Try subprocess; reduce test matrix * Add `shell=True` * More `describe`s * Correct `kubectl --selector` commands * Reflect r/ENTRYPOINT/CMD in Dockerfile * Try matrix * Debugging Kubenernetes 1.16 * Additional MicroK8s 1.16+1.17 and debugging * Generate: `*-webhook-log` from GitHub Actions * Add some debugging statements * K8s 1.16 expects AdmissionReview to be v1beta1 * Checking K3s 1.16+1.17 * Ordering may be significant * Reverted * Don't delete CRDs; Remove redundant debugging * Limit to distro-versions known to work for E2Es * Bumped OpenAPI-generated sources due to Akri security audit issues * Documents `run-webook.py` * Typo * `expect` only when providing useful message * fix Makefile formatting for image push [SAME VERSION] * add updates to lock file fir admission controller webhook dependencies (#254) * proposal for logging standards * Fix typo * Simplify containers makefile (#256) * simplify makefile for containers * add for all containers * fix indent * bump version * try to fix indentation * fix manifest creation error (#257) * Fix onvif * changes to the particular makefiles should invoke container workflow (#258) * Add more docs for akri build (#260) * add docs for akri build * address feedback * add local instructions * fix mistakes * Auto-updated dependencies * Allow dependencies auto update to be manually run * Auto-updated dependencies * increase timeout for tarpaulin and check-rust workflows (#266) * Increase docker create timeout to 600 minutes and document this hidden timeout (#267) * Try same env var for action and container timeout * output script to verify * try reusing timeout-minutes directly * Try moving timeout to step and access that directly * try fixing env var creation * try using env var again https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#job-context suggests timeout-minutes can be set from env * Use step.timeout-minutes because it can pull from env var * Update run-tarpaulin.yml * just make the docker keep alive script sleep 600 minutes * remove env var * fix typo * typos, typos, typos * create certs for webhook e2e tests with longer lasting CA (#271) * Enable Discovery Handlers as Pods (#252) * Move security audit checks to a daily schedule instead of on PRs * Fix typo * Link to specific github run url from github context * rename rust jobs * remove unnecessary job * only run on main and not forks * Update security-audit.yml Restrict the action to main repo and not forks * Serialize Configuration Properties for Webhook (#276) * don't skip serializing configuration properties * increase version * rust fmt fixes * Try to ONLY build rust once per platform (#270) * build rust once/platform, share with artifacts * timeout => timeout-minutes * trigger when build-rust-code changes * only run container builds after successful rust build * install rust toolchain to build rust * add crossbuild qemu * reduce retention to minimum * make test timeout longer * update version * Fix artifact name [SAME VERSION] (#280) * fix artifact name [SAME VERSION] * only run for main * Add Workflow to Cancel Previous PR Workflows [SAME VERSION] (#281) * add workflow to cancel all previous workflows on a pr * remove security workflow and cancel workflow from list * Build both slim and full Agents [IGNORE INTERMEDIATE BUILDS] (#279) * specify component name in agent multiarch build (#285) * Build discovery handler images [SAME VERSION] (#286) * Enable Canceling Previous Workflows on PRs from Forks [SAME VERSION] (#284) * use workflow_run trigger in cancel workflow * only run cancel workflow explicitly for pull_request events * Increase e2e test timeout [SAME VERSION] (#287) * increase e2e test timeout * increase timeout further * Ensure Discovery Handler is marked as available again after Configuration deletions (#282) * Make sure `shared` environment variable is set in debug echo Discovery Handler (#289) * Overwrite latest tag when user specifies a tag (#293) * overwrite latest tag when user supplies tag * increase version * Auto-updated dependencies * Reference correct Agent features for embedding Discovery Handlers [IGNORE INTERMEDIATE BUILDS] (#295) * Debug echo properties (#288) * Add Documentation on using Debug Echo [SAME VERSION] (#296) * make sure webhook-configuration version updated in lock file (#300) * Remove un-utilized CRD field (#290) * remove rbac property from Instance CRD * bump minor version to 0.5.0 for breaking change * Rename CRD fields to be more descriptive (#297) * change Configuration.properties to Configuration.brokerProperties * change Instance.metadata to Instance.brokerProperties * increase minor version due to CRD breaking change * fix errors in user guide (#302) * Remove udev directory mount from Agent (#304) * Extend documentation (#303) * Documentation fixes (#307) A few simple fixes to the Debug Echo documentation. Signed-off-by: Hernan Gatta <[email protected]> * fix opc ua configuration table in docs (#309) * use sudo when getting logs in e2e tests (#312) * suggest background reading before discovery handler development (#310) * Update cluster setup documentation (#308) * lint helm chart with values (#306) * Add documentation on setting a Configuration's `brokerProperties` (#311) * Fix end-to-end tests (#301) * Reduce K3s and MicroK8s eviction limits for tests [SAME VERSION] (#313) * reduce K3s eviction limits for tests * reduce MicroK8s limits * Rename configuration templates (#315) * rename configuration templates * increase version * Set compute resource limits and requests for all Akri components (#305) * Extensibility example fixes [SAME VERSION] (#321) * Use latest Rust and Tarpaulin (#318) * use latest tarpaulin in workflow * use latest rust in workflows * increase version * use previous tarpaulin version due to segfault issue * Increase dependency minor versions where possible [SAME VERSION] (#323) * increase cfg-if, env_logger, mockall_double, prometheus, udev, opcua-client, and rand crate minor versions * increase version * extend sleep in test case workflow to 45 minutes for component builds to complete (#326) * Test e2e tests on latest K8s version [SAME VERSION] (#317) * reference Akri's release Helm chart in documentation (#325) * Update CHANGELOG for v0.6.5 release (#319) * [Extensibility] HTTP protocol (branch: http-extensibility) (#135) * Initial commit * Working * Correct errors & revise for Device|Discovery v2 * Working * Should (!) work * Typo * Enrich * Refine * Fixes: #102 * Moves * Resolved * Revised for `dnsPolicy` * Dockerfiles * Added gRPC solution * Replaced * Kubernetes specs * Merge * Correct Kubernetes config filenames * Typo * Removed duplicate * Standalone broker exposes no ports * end-to-end-demo-gce.md: fixing 1 typo (#131) Hi, just removing the superfluous 's' Didier * link to codecov.io and use codecov action (#134) * Update end-to-end-demo.md (#112) * Update end-to-end-demo.md Hi there I extended the documentatio with the installation of linux-modules-extra because I had to add this package on Ubuntu 20.04 LTS (on a GCE instance): it seems that it doesn't come by default on distros / images aimed at backend servers. v4l loopback won't load if this other module is not present. Didier * Update end-to-end-demo.md fixing initial text according to revision notes in PR review * Update end-to-end-demo.md fixing style * Update end-to-end-demo.md simplifying text, based on PR review by Kate * rebase to finalize PR * Fix codecov for pull_request_target [SAME VERSION] (#138) * fix codecov for pull_request_target * add anti-pull-request-* conditional * Do not upload to codecov on PRs (#140) * end-to-end-demo-rpi4: fixing 1 typo (#143) Hello, Title says it all Didier * end-to-end demo : fixing 1 typo (#141) Hi there, The title says it all Didier * Minor e2e demo doc flow change (#139) * Minor flow change * Cleanup * Expand udev rule support (#130) * add support for SUBSYSTEMS, KERNELS, ATTRIBUTES, DRIVERS, and TAGS fields * fix formatting and add comments * add recursive search for subsystems * update udev configuration document with newly supported fields * check child device for field before looking up hierarchy * update version * document how to filter for only devices with devnodes when testing * reuse shared code * rust fmt fixes * add inline documentation for regex matching * Update end-to-end-demo.md (#113) Hi there, I added a comment regarding the importance of the version of v4l2loopback to avoid my personal initial mistake to others: I did not realize at start that the version was so important. So I went through this ticket with the dev: v4l2loopback/v4l2loopback#370 Didier Co-authored-by: bfjelds <[email protected]> * Add sudo into udev configuration doc (#144) * Use arm7 * Reference arm7 * Reference branch on clone * `REPO="akri"` * Helm template for Agent already revised * Consistency * Use `./version.txt` for Akri version * Automate IMAGE updates * Build|Push gRPC images; Add'l consistency * Add Action to build HTTP Broker containers * I guess `http-extensibility` not `main` * Enable PRs to test changes to helm charts [SAME VERSION] (#146) * enable PRs to test changes to helm charts * handle manual job, fix push and release conditional * trigger e2e for helm changes * Updates references to `armv7` * Removed `shared` from HTTP Broker Dockerfiles * Correct version reference for image tags * Typo * Adds Golang Device|Discovery and alt-gRPC * s/println!/trace! * Consistency * Reflect relocation of apps * Simplify environment * Typo * Consistency on repo names for HTTP apps * Rename HTTP apps gRPC images * GitHub Actions to build HTTP apps containers * Configure broker security context (#148) * make udev broker security context configurable * update version * change group permissions to be default in example * fix comment with group id * Remove privileged security context from Akri Agent (#147) * Remove (redundant!?) aliasing * Remove healthchecks (`/healthz`) * Removed `akri-shared` dep * Simplified using `strategy.matrix` * Revised to use `strategy.matrix` * reference macro using fully-qualified module name * r|Microsoft|deislabs/akri * Remove unnecessary include * Add precision on arch being built * Consistent image naming and v1 * Remove `/v2/` references * Explantion of different broker implementations * Cleanup Workflows [IGNORE INTERMEDIATE BUILDS][SAME VERSION] (#155) * don't persist github creds and remove setup.sh from workflows * Update CHANGELOG for release v0.0.44 (#153) * Update CHANGELOG for release v0.0.44 * update release to better describe helm changes and supported flavors * add note about known k3s incompat * Pin CHANGELOG changes to the expected label * Fix CHANGELOG typo * Optimize build and test all supported versions (#157) * build once, test all supported versions * test step needs build step * fix copy-paste error * use startsWith instead of '==' to check kube-runtime * for download, specifying path introduces an unwanted directory * load the artifact containers into docker for Kubernetes * microk8s ctr images import is mysteriously failing with 'context deadline exceeded' * avoid k3s' crictl, add more testing versions for kubernetes & k3s * allow downgrades for kubernetes install, try to understand microk8s image import failure * swallow the return code of version checks * use persist-creds=false * disable microk8s 1.19 for now * microk8s 1.19 requires small sleep before it is ready for ctr requests * to ensure sequential running, we cannot skip the build-containers job (just its steps) (#159) * Update user guide with first release tag for Akri release chart (#160) * update user guide with first release tag for akri release chart * reference akri release chart instead of dev chart in docs * Remove pull_request_target triggers and minimize credential use [SAME VERSION] [IGNORE INTERMEDIATE BUILDS] (#161) * remove dockerhub credentials from workflows * add link to repo to intermediate containers * remove pull_request_target * only log into docker when pushing changes to ghcr * Typos * Added existing "Extensibility" preamble * Tidy * Replace "Extensibility" * attempt to merge extensibility.md i focused on the standalone to cut down on the size (the file is still 800+ lines) with a thought towards linking to some md files in http-extensibility branch for more details (on the rust grpc and golang grpc). * tighten up to reduce size combine device and discovery into one http server/pod, remove some comments from quoted code * provide a few more explanations * fix typo * Use matrix for build-http-broker-containers.yaml * Replace backup md file with extensibility-http-grpc.md Create document for deeper dive into gRPC broker/client code. This can be referenced from extensibility.md. * update extensibility.md to be consistent 'HTTP' * create links to http-extensibility branch * after walkthrough, make a couple fixes * use different curl container busyboxplus seems to have a dns problem that manifests on k3s * remove workflows for http-extensibility * remove new workflows in extensibility branch Co-authored-by: Didier Durand <[email protected]> Co-authored-by: bfjelds <[email protected]> Co-authored-by: Jiri Appl <[email protected]> Co-authored-by: Kate Goldenring <[email protected]> * add http discovery handler to the discovery handler modules folder Co-authored-by: Didier Durand <[email protected]> Co-authored-by: Jiayi Hu <[email protected]> Co-authored-by: Jiayi Hu <[email protected]> Co-authored-by: Roaa <[email protected]> Co-authored-by: Daz Wilkin <[email protected]> Co-authored-by: Ronan Flynn-Curran <[email protected]> Co-authored-by: romoh <[email protected]> Co-authored-by: Chao Zheng <[email protected]> Co-authored-by: bfjelds <[email protected]> Co-authored-by: Jiri Appl <[email protected]> Co-authored-by: Hernan Gatta <[email protected]>

* test latest k8s versions [SAME VERSION] (project-akri#188) * update changelog for 0.1.5 release (project-akri#189) * add more description and simplify commands in e2e demo (project-akri#191) * Various typos fixed on new OPCUA documentation (project-akri#195) * Various typos fixed on new OPCUA documentation Signed-off-by: Didier Durand <[email protected]> * Fixes on extensibility documentation Signed-off-by: Didier Durand <[email protected]> * Make end-to-end demo on Raspberry Pi simpler and more descriptive (project-akri#193) * mirror updates in e2e demo in rasp-pi demo * syntax changes, capacity specification, building kernel module * fix syntax issues * Get Agent logs before testing their contents [SAME VERSION] (project-akri#197) * add additional checks to see why getting logs failing * log information when get crictl error * use os.system to log info * try using shell with subprocess when doing complex commands * Fix failing to query container runtime in test workflow on MicroK8s [SAME VERSION] (project-akri#202) * loop query containerd several times instead of sleeping * shorten agent log query * Allow overwriting controller.nodeSelectors (project-akri#194) * Allow overwriting controller.nodeSelectors * Add note about `sed` for MacOS contributors * Allow conditional compilation of agent protocols (project-akri#196) * Allow conditional compilation of agent protocols * Bump patch version Co-authored-by: Jiayi Hu <[email protected]> * Prometheus (project-akri#190) * add frame count metric to udev broker * add prometheus metrics * format metrics.rs * update Helm charts for Prometheus support * Prometheus documentation * increase supported rust version to latest * fix metrics chart * use PodMonitor instead of ServiceMonitor for Akri Agent and Controller * update tarpaulin to latest version to fix protobuf issue * remove debug echo shared setting * update version * Use anyhow instead of the deprecated failure crate (project-akri#201) * Switch to use anyhow crate instead of failure crate Include Cargo.lock updates * Rebase and increment patch version * Merge conflict * Add missing version updates after rebase * [Helm] Added CRD variables (project-akri#205) * Added CRD variables * `version.sh` updates Helm `crds.version` * Bump version * Document naming convention (project-akri#204) * Initial commit * Remove references to `http` example * Added `udev-broker` * Added `akri-udev` * Rephrased * Typo * naming-guidelines >> development.md * r/udev-broker/udev-video-broker * punctuation * r/Resources/resources * r/Compute/Computer (project-akri#210) * fix instructions for cloning and building v4l2 (project-akri#209) * Update hyper version and other dependencies * Update patch version * Explicitly specify h2 patch version dependency to avoid future accidentl updates by cargo-update or similar * Remove redundant unnecessary hyper dependencies * Add security audit workflow * Move security audit to existing check-rust workflow * Simplify mockall usage (project-akri#214) * Simplify mockall usage * Bump patch version * add project logo and art [SAME VERSION] (project-akri#149) * add akri branding and artwork Signed-off-by: flynnduism <[email protected]> * edit artwork README Signed-off-by: flynnduism <[email protected]> * re-render dark logos Signed-off-by: flynnduism <[email protected]> * redraw the svg icon to fix glitchy render Signed-off-by: flynnduism <[email protected]> * update logo.svg Signed-off-by: flynnduism <[email protected]> * fix the mixed up color hex values Signed-off-by: flynnduism <[email protected]> * update branding with inconsolata typeface Signed-off-by: flynnduism <[email protected]> * use correct casing for mentions of Akri Signed-off-by: flynnduism <[email protected]> * remove default documentation to use latest containers (project-akri#218) * Image pull policy (project-akri#207) * change default image pull policy to IfNotPresent * set ImagePullPolicy to default to empty * increase version * only add ImagePullPolicy if specified * Add a workflow to automated dependencies update * fix the instruction of the end to end demo and the rspi4 demo. (project-akri#217) * Add comment for cargo-audit action behavior * use akri bot PAT for the autoupdate workflow * update token secret name * Patch anomaly detection app (project-akri#229) * close connection to grpc service each loop * update version * use local subchannel pool * fix title typo * Add logo to README [SAME VERSION] (project-akri#220) * Auto-updated dependencies * Auto-updated dependencies * Allow unreachable code introduced with feature-driven protocols [SAME VERSION] (project-akri#232) * Allow unreachable code introduced with feature-driven protocols [SAME VERSION] The catchall match is there in case someone turns off one of the handlers via features ... but at runtime the disabled feature is configured. * update for cargo fmt * add comment explanation * fix cargo fmt * Improve development experience with timestamped container labels (project-akri#234) Using labels based on version.txt can make it difficult to iterate during development. Creating timestamped labels for the local build experience should make it easier to differentiate between development builds. * Reduce frequency of dependency update [SAME VERSION] (project-akri#236) * Reduce frequency of dependency update Our dependencies update on a daily basis ... not sure it makes sense for us to update our version each time someone else's do. I chose a monthly cadence because it would, at most, cause us to update our version 12x per year, which doesn't seem too bad. * Update auto-update-dependencies.yml * update code owners * Lazy eval of makefile timestamp breaks local build [SAME VERSION] [IGNORE INTERMEDIATE BUILDS] (project-akri#242) * Update hyper crate for security risk * patch version update * force override kubernetes package to deal with installation errors * Fix typo * Simplify matrix; Use all versions 1.16+ for all distros (project-akri#244) * Simplify matrix; Use all versions 1.16+ for all distros * Remove non-existent webhook test * Remove `microk8s.enable helm3` appears redundant and fails * Link Kubernetes and crictl for K3s|MicroK8s * Per: project-akri#206 (comment) * Typo * fix matrix and crictl path * Consistency and double-quote to expand `${PWD}` * Temporarily (!) block K3s 1.16 due to hard-coded device plugs (k3s-io/k3s#1390) * Test mapping K3s 1.16 device plugins to default K8s location * Create kubectl path in expected location if not present * Consistent w/ MicroK8s step * Wait until cluster has stabilized before wrangling device-plugins * Initial documentation for `run-test-cases.yml` * Correct indentation Co-authored-by: bfjelds <[email protected]> * add microk8s inspect to installation * try waiting repeatedly for microk8s to be ready * Cleanup streaming app sample (project-akri#173) * Initial version * Create logical classes * Fixes and optimizations * Use naming guideline for method names * Version bump * Version bump Co-authored-by: Brian Fjeldstad <[email protected]> * Admission Controller (Webhook) for Akri Configuration(s) (project-akri#206) * Helm Chart updates * Add 'webhooks' to the project * Commented Helm Chart `webhook-configuration` * Template `webhook-configuration` * Webhook service * Dockerfile (draft) * GitHub Action (draft) * Force build * Trying again * And again * Added GHCR secret to fork * Grrr * Grrr * Issue w/ GitHub Action build * Undo * Consistency * Revised Rust Webhook to not require these * Preliminary README * Update `version.sh` and run `./version.sh -u -s` * Added `webhook-configuration` * Does this resolve Helm Linting concern? * r/AMD32V7/AMR32V7 * Are these required? * Follow pattern * Follow Akri pattern * r/webhook-configuration/webhookConfiguration * correct path references * Cross-build Rust * Exceptions no longer required * Assume `./target/release/webhook-configuration` * Expose x-plat targets to docker * Omission * Improve handling of flags * Tidy * Correct indendation * Version bump * Update `CHANGELOG` * `cargo.lock` versions now bumped too * Temporarily (!) bumping build timeouts * Corrected component reference * Missed `build-arg=CROSS_BUILD_TARGET=...` * Avoid Helm issue with un-`named` `Lists` * Revert `CHANGELOG` * Added unit tests * Filters Akri Configuration manifest so that it validates * Added test for filtered `generation` * Added test for extended (complete) Config * To avoid GitHub Actions build failure * End-to-End Webhook tests * Integration tests require `actix-rt` * Include `webhook-configuration` container image * Learns all the things! * Typo * Added `webhook` to `test-case`s * Removed `get_agent_and_controller` reference * Simplified Actions `matrix`; Updated Webhook certs * Override `webhookConfiguration.image.tag` to `pr-amd64` * Add debugging * Typo * sudo'ing `kubectl` commands * Try subprocess; reduce test matrix * Add `shell=True` * More `describe`s * Correct `kubectl --selector` commands * Reflect r/ENTRYPOINT/CMD in Dockerfile * Try matrix * Debugging Kubenernetes 1.16 * Additional MicroK8s 1.16+1.17 and debugging * Generate: `*-webhook-log` from GitHub Actions * Add some debugging statements * K8s 1.16 expects AdmissionReview to be v1beta1 * Checking K3s 1.16+1.17 * Ordering may be significant * Reverted * Don't delete CRDs; Remove redundant debugging * Limit to distro-versions known to work for E2Es * Bumped OpenAPI-generated sources due to Akri security audit issues * Documents `run-webook.py` * Typo * `expect` only when providing useful message * fix Makefile formatting for image push [SAME VERSION] * add updates to lock file fir admission controller webhook dependencies (project-akri#254) * proposal for logging standards * Fix typo * Simplify containers makefile (project-akri#256) * simplify makefile for containers * add for all containers * fix indent * bump version * try to fix indentation * fix manifest creation error (project-akri#257) * Fix onvif * changes to the particular makefiles should invoke container workflow (project-akri#258) * Add more docs for akri build (project-akri#260) * add docs for akri build * address feedback * add local instructions * fix mistakes * Auto-updated dependencies * Allow dependencies auto update to be manually run * Auto-updated dependencies * increase timeout for tarpaulin and check-rust workflows (project-akri#266) * Increase docker create timeout to 600 minutes and document this hidden timeout (project-akri#267) * Try same env var for action and container timeout * output script to verify * try reusing timeout-minutes directly * Try moving timeout to step and access that directly * try fixing env var creation * try using env var again https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#job-context suggests timeout-minutes can be set from env * Use step.timeout-minutes because it can pull from env var * Update run-tarpaulin.yml * just make the docker keep alive script sleep 600 minutes * remove env var * fix typo * typos, typos, typos * create certs for webhook e2e tests with longer lasting CA (project-akri#271) * Enable Discovery Handlers as Pods (project-akri#252) * Move security audit checks to a daily schedule instead of on PRs * Fix typo * Link to specific github run url from github context * rename rust jobs * remove unnecessary job * only run on main and not forks * Update security-audit.yml Restrict the action to main repo and not forks * Serialize Configuration Properties for Webhook (project-akri#276) * don't skip serializing configuration properties * increase version * rust fmt fixes * Try to ONLY build rust once per platform (project-akri#270) * build rust once/platform, share with artifacts * timeout => timeout-minutes * trigger when build-rust-code changes * only run container builds after successful rust build * install rust toolchain to build rust * add crossbuild qemu * reduce retention to minimum * make test timeout longer * update version * Fix artifact name [SAME VERSION] (project-akri#280) * fix artifact name [SAME VERSION] * only run for main * Add Workflow to Cancel Previous PR Workflows [SAME VERSION] (project-akri#281) * add workflow to cancel all previous workflows on a pr * remove security workflow and cancel workflow from list * Build both slim and full Agents [IGNORE INTERMEDIATE BUILDS] (project-akri#279) * specify component name in agent multiarch build (project-akri#285) * Build discovery handler images [SAME VERSION] (project-akri#286) * Enable Canceling Previous Workflows on PRs from Forks [SAME VERSION] (project-akri#284) * use workflow_run trigger in cancel workflow * only run cancel workflow explicitly for pull_request events * Increase e2e test timeout [SAME VERSION] (project-akri#287) * increase e2e test timeout * increase timeout further * Ensure Discovery Handler is marked as available again after Configuration deletions (project-akri#282) * Make sure `shared` environment variable is set in debug echo Discovery Handler (project-akri#289) * Overwrite latest tag when user specifies a tag (project-akri#293) * overwrite latest tag when user supplies tag * increase version * Auto-updated dependencies * Reference correct Agent features for embedding Discovery Handlers [IGNORE INTERMEDIATE BUILDS] (project-akri#295) * Debug echo properties (project-akri#288) * Add Documentation on using Debug Echo [SAME VERSION] (project-akri#296) * make sure webhook-configuration version updated in lock file (project-akri#300) * Remove un-utilized CRD field (project-akri#290) * remove rbac property from Instance CRD * bump minor version to 0.5.0 for breaking change * Rename CRD fields to be more descriptive (project-akri#297) * change Configuration.properties to Configuration.brokerProperties * change Instance.metadata to Instance.brokerProperties * increase minor version due to CRD breaking change * fix errors in user guide (project-akri#302) * Remove udev directory mount from Agent (project-akri#304) * Extend documentation (project-akri#303) * Documentation fixes (project-akri#307) A few simple fixes to the Debug Echo documentation. Signed-off-by: Hernan Gatta <[email protected]> * fix opc ua configuration table in docs (project-akri#309) * use sudo when getting logs in e2e tests (project-akri#312) * suggest background reading before discovery handler development (project-akri#310) * Update cluster setup documentation (project-akri#308) * lint helm chart with values (project-akri#306) * Add documentation on setting a Configuration's `brokerProperties` (project-akri#311) * Fix end-to-end tests (project-akri#301) * Reduce K3s and MicroK8s eviction limits for tests [SAME VERSION] (project-akri#313) * reduce K3s eviction limits for tests * reduce MicroK8s limits * Rename configuration templates (project-akri#315) * rename configuration templates * increase version * Set compute resource limits and requests for all Akri components (project-akri#305) * Extensibility example fixes [SAME VERSION] (project-akri#321) * Use latest Rust and Tarpaulin (project-akri#318) * use latest tarpaulin in workflow * use latest rust in workflows * increase version * use previous tarpaulin version due to segfault issue * Increase dependency minor versions where possible [SAME VERSION] (project-akri#323) * increase cfg-if, env_logger, mockall_double, prometheus, udev, opcua-client, and rand crate minor versions * increase version * extend sleep in test case workflow to 45 minutes for component builds to complete (project-akri#326) * Test e2e tests on latest K8s version [SAME VERSION] (project-akri#317) * reference Akri's release Helm chart in documentation (project-akri#325) * Update CHANGELOG for v0.6.5 release (project-akri#319) * [Extensibility] HTTP protocol (branch: http-extensibility) (project-akri#135) * Initial commit * Working * Correct errors & revise for Device|Discovery v2 * Working * Should (!) work * Typo * Enrich * Refine * Fixes: project-akri#102 * Moves * Resolved * Revised for `dnsPolicy` * Dockerfiles * Added gRPC solution * Replaced * Kubernetes specs * Merge * Correct Kubernetes config filenames * Typo * Removed duplicate * Standalone broker exposes no ports * end-to-end-demo-gce.md: fixing 1 typo (project-akri#131) Hi, just removing the superfluous 's' Didier * link to codecov.io and use codecov action (project-akri#134) * Update end-to-end-demo.md (project-akri#112) * Update end-to-end-demo.md Hi there I extended the documentatio with the installation of linux-modules-extra because I had to add this package on Ubuntu 20.04 LTS (on a GCE instance): it seems that it doesn't come by default on distros / images aimed at backend servers. v4l loopback won't load if this other module is not present. Didier * Update end-to-end-demo.md fixing initial text according to revision notes in PR review * Update end-to-end-demo.md fixing style * Update end-to-end-demo.md simplifying text, based on PR review by Kate * rebase to finalize PR * Fix codecov for pull_request_target [SAME VERSION] (project-akri#138) * fix codecov for pull_request_target * add anti-pull-request-* conditional * Do not upload to codecov on PRs (project-akri#140) * end-to-end-demo-rpi4: fixing 1 typo (project-akri#143) Hello, Title says it all Didier * end-to-end demo : fixing 1 typo (project-akri#141) Hi there, The title says it all Didier * Minor e2e demo doc flow change (project-akri#139) * Minor flow change * Cleanup * Expand udev rule support (project-akri#130) * add support for SUBSYSTEMS, KERNELS, ATTRIBUTES, DRIVERS, and TAGS fields * fix formatting and add comments * add recursive search for subsystems * update udev configuration document with newly supported fields * check child device for field before looking up hierarchy * update version * document how to filter for only devices with devnodes when testing * reuse shared code * rust fmt fixes * add inline documentation for regex matching * Update end-to-end-demo.md (project-akri#113) Hi there, I added a comment regarding the importance of the version of v4l2loopback to avoid my personal initial mistake to others: I did not realize at start that the version was so important. So I went through this ticket with the dev: v4l2loopback/v4l2loopback#370 Didier Co-authored-by: bfjelds <[email protected]> * Add sudo into udev configuration doc (project-akri#144) * Use arm7 * Reference arm7 * Reference branch on clone * `REPO="akri"` * Helm template for Agent already revised * Consistency * Use `./version.txt` for Akri version * Automate IMAGE updates * Build|Push gRPC images; Add'l consistency * Add Action to build HTTP Broker containers * I guess `http-extensibility` not `main` * Enable PRs to test changes to helm charts [SAME VERSION] (project-akri#146) * enable PRs to test changes to helm charts * handle manual job, fix push and release conditional * trigger e2e for helm changes * Updates references to `armv7` * Removed `shared` from HTTP Broker Dockerfiles * Correct version reference for image tags * Typo * Adds Golang Device|Discovery and alt-gRPC * s/println!/trace! * Consistency * Reflect relocation of apps * Simplify environment * Typo * Consistency on repo names for HTTP apps * Rename HTTP apps gRPC images * GitHub Actions to build HTTP apps containers * Configure broker security context (project-akri#148) * make udev broker security context configurable * update version * change group permissions to be default in example * fix comment with group id * Remove privileged security context from Akri Agent (project-akri#147) * Remove (redundant!?) aliasing * Remove healthchecks (`/healthz`) * Removed `akri-shared` dep * Simplified using `strategy.matrix` * Revised to use `strategy.matrix` * reference macro using fully-qualified module name * r|Microsoft|deislabs/akri * Remove unnecessary include * Add precision on arch being built * Consistent image naming and v1 * Remove `/v2/` references * Explantion of different broker implementations * Cleanup Workflows [IGNORE INTERMEDIATE BUILDS][SAME VERSION] (project-akri#155) * don't persist github creds and remove setup.sh from workflows * Update CHANGELOG for release v0.0.44 (project-akri#153) * Update CHANGELOG for release v0.0.44 * update release to better describe helm changes and supported flavors * add note about known k3s incompat * Pin CHANGELOG changes to the expected label * Fix CHANGELOG typo * Optimize build and test all supported versions (project-akri#157) * build once, test all supported versions * test step needs build step * fix copy-paste error * use startsWith instead of '==' to check kube-runtime * for download, specifying path introduces an unwanted directory * load the artifact containers into docker for Kubernetes * microk8s ctr images import is mysteriously failing with 'context deadline exceeded' * avoid k3s' crictl, add more testing versions for kubernetes & k3s * allow downgrades for kubernetes install, try to understand microk8s image import failure * swallow the return code of version checks * use persist-creds=false * disable microk8s 1.19 for now * microk8s 1.19 requires small sleep before it is ready for ctr requests * to ensure sequential running, we cannot skip the build-containers job (just its steps) (project-akri#159) * Update user guide with first release tag for Akri release chart (project-akri#160) * update user guide with first release tag for akri release chart * reference akri release chart instead of dev chart in docs * Remove pull_request_target triggers and minimize credential use [SAME VERSION] [IGNORE INTERMEDIATE BUILDS] (project-akri#161) * remove dockerhub credentials from workflows * add link to repo to intermediate containers * remove pull_request_target * only log into docker when pushing changes to ghcr * Typos * Added existing "Extensibility" preamble * Tidy * Replace "Extensibility" * attempt to merge extensibility.md i focused on the standalone to cut down on the size (the file is still 800+ lines) with a thought towards linking to some md files in http-extensibility branch for more details (on the rust grpc and golang grpc). * tighten up to reduce size combine device and discovery into one http server/pod, remove some comments from quoted code * provide a few more explanations * fix typo * Use matrix for build-http-broker-containers.yaml * Replace backup md file with extensibility-http-grpc.md Create document for deeper dive into gRPC broker/client code. This can be referenced from extensibility.md. * update extensibility.md to be consistent 'HTTP' * create links to http-extensibility branch * after walkthrough, make a couple fixes * use different curl container busyboxplus seems to have a dns problem that manifests on k3s * remove workflows for http-extensibility * remove new workflows in extensibility branch Co-authored-by: Didier Durand <[email protected]> Co-authored-by: bfjelds <[email protected]> Co-authored-by: Jiri Appl <[email protected]> Co-authored-by: Kate Goldenring <[email protected]> * add http discovery handler to the discovery handler modules folder Co-authored-by: Didier Durand <[email protected]> Co-authored-by: Jiayi Hu <[email protected]> Co-authored-by: Jiayi Hu <[email protected]> Co-authored-by: Roaa <[email protected]> Co-authored-by: Daz Wilkin <[email protected]> Co-authored-by: Ronan Flynn-Curran <[email protected]> Co-authored-by: romoh <[email protected]> Co-authored-by: Chao Zheng <[email protected]> Co-authored-by: bfjelds <[email protected]> Co-authored-by: Jiri Appl <[email protected]> Co-authored-by: Hernan Gatta <[email protected]>

kate-goldenring requested review from bfjelds, Britel, jiria and romoh as code owners February 23, 2021 21:34

bfjelds reviewed Feb 23, 2021

View reviewed changes

.dockerignore Outdated Show resolved Hide resolved

bfjelds reviewed Feb 23, 2021

View reviewed changes

test/yaml/akri-opcua.yaml Outdated Show resolved Hide resolved

bfjelds reviewed Feb 23, 2021

View reviewed changes

test/yaml/akri-onvif-video.yaml Show resolved Hide resolved

bfjelds reviewed Feb 23, 2021

View reviewed changes

test/yaml/akri-debug-echo-foo.yaml Outdated Show resolved Hide resolved

bfjelds reviewed Feb 23, 2021

View reviewed changes

shared/src/lib.rs Outdated Show resolved Hide resolved

kate-goldenring commented Feb 23, 2021

View reviewed changes

agent/Cargo.toml Outdated Show resolved Hide resolved

kate-goldenring commented Feb 23, 2021

View reviewed changes

docs/extensibility.md Show resolved Hide resolved

kate-goldenring commented Feb 23, 2021

View reviewed changes

bfjelds reviewed Feb 23, 2021

View reviewed changes

romoh reviewed Feb 23, 2021

View reviewed changes

bfjelds reviewed Feb 23, 2021

View reviewed changes

deployment/helm/templates/custom-discovery.yaml Outdated Show resolved Hide resolved

romoh reviewed Feb 23, 2021

View reviewed changes

Cargo.toml Show resolved Hide resolved

bfjelds reviewed Feb 23, 2021

View reviewed changes

deployment/helm/values.yaml Show resolved Hide resolved

bfjelds reviewed Feb 23, 2021

View reviewed changes

discovery-handler-modules/debug-echo-discovery-handler/src/main.rs Outdated Show resolved Hide resolved

romoh reviewed Feb 23, 2021

View reviewed changes

agent/src/util/discovery_operator.rs Show resolved Hide resolved