Skip to content

Release Notes

Here we document all major changes in a release, incl. added and deprecated functionalities. In particular, we provide a comprehensive list of system-breaking changes that could affect end users.

Breaking Changes

Version number Change How to fix

[17.04.2024] v0.0.32

Description

This release enhances Eurodat web frontend and service documentation.

Added features

  • Keycloak for EuroDat web frontend has been configured
  • Hardening of the web container has been implemented
  • Contract-service has been implemented
  • Automatic validation of workflows and workflow templates (names, format, etc.) during dynamic app registration has been implemented
  • Copy-and-Paste-Logic for MiNaBo app has been added
  • Documentation of services has been updated
  • Clusters are now deployed with an own artifact registry for UC images

[ 04.04.2024 ] v0.0.31

Description

This release enhances the dynamic app registration process and hardcodes a new Minabo app.

Added features

  • Minabo app now has a Hello World workflow
  • Dynamic app registration now registers a workflow template when registering an app
  • app-of-apps helm chart is now part of the trustee-plattform repository
  • Enhanced app-of-apps helm charts to support GCP private cluster deployment

[ 20.03.2024 ] v0.0.30

Description

This release provides bugfixes, improvements to the development workflow and maintainability. In addition, we started working on enabling service providers to register new apps. Further, we have started integrating Keycloak as an identity provider for the user journey login process.

Added features

  • Added creation of database roles when registering a new app
  • Deduplicated cloud provider versions of app-of-apps helm chart
  • Deduplicated stages of app-of-apps helm chart
  • Integrated Keycloak in Angular login component

[ 08.03.2024 ] v0.0.29

Description

This release now provides an API to dynamically register new data apps which does not require code changes within the repository or an updated release to get a new schema (safe deposit box / transactional databases) and app definition (Argo Workflow Template) into EuroDaT workflow engine. We further continued to work on the onboarding frontend which now integrates into a dummy EuroDaT homepage.

Added features

  • Added login/sign-up options (dummy only) to the EuroDaT homepage (dummy only) for participant onboarding
  • Added API to dynamically register data apps

[ 20.02.2024 ] v0.0.28

Description

This release focuses mainly on bug fixes concerning deployment stability and further code refactoring to a microservice architecture. To take care of the participant onboarding we have started to work on the EuroDaT platform web frontend.

Added features

  • We have started to remove obsolete broker appearances
  • The sceletal structure for participant onboarding web frontend has been created

[ 06.02.2024 ] v0.0.27

Description

This release includes new features which improve the permission management and updates for more security by using an image digest.

Added features

  • We increased security by using an image digest to refer to images to ensure that the image that is pulled or pushed is the exact one we intend to push or pull.
  • The AppRepository is split into its own microservice for a cleaner architecture.
  • To ensure proper permission management for various apps, we added an explicit specification for the participants of every transaction as a prerequisite to check permissions.
  • Fixed bugs for the nightly pipeline.

[ 24.01.2024 ] v0.0.26

Description

A small release that adds backmessaging service features and some bug fixes

Added features

  • Transaction participants are now informed by a backmessage if a transaction or workflow is started
  • The image registry for the client controller is set properly
  • ArgoWorkflows mock pods are now cleaned up after completion without errors
  • Fixed a bug where the backmessaging service helm chart was not published in a release

[ 12.01.2024 ] v0.0.25

Description

A small release with refactorings and stabilisation changes

Added features

  • safeAML and UC4 Safe Deposit Box Schema Update
  • UC2 workflow template fix
  • The RabbitMQ user from the container can now access the queues of all transaction participants
  • Simultanious calls in user resource are now handled correctly

[ 19.12.2023 ] v0.0.24

Description

A small release with refactorings and stabilisation changes

Added features

  • UC4 Safe Deposit Box Schema Update
  • Release pipeline is a full pipeline now
  • Microservice use Flyway to manage their schemas
  • Automated test checks that all EuroDaT endpoints require authentication

Breaking Changes

Version number Change How to fix
v0.0.24 Postgres instances merged Run reset_db.sh

[ 12.12.2023 ] v0.0.23

Description

ArgoCD is used for all non-local deployments of EuroDaT to a cluster, replacing the previous Helm CLI deployment to development environments. EuroDaT now supports dynamic registration of clients together with their client selector, replacing the previous hardcoding. Tempo and Prometheus were introduced to enhance monitoring of the cluster. EuroDaT onboarding documentation has received a great overhaul. Kubernetes was updated to 1.27.7. OpenCost has been removed from EuroDaT. EuroDaT now provides a QA-INT environment with the official latest release.

Added features

  • Dynamic client registration

Breaking Changes

Version number Change How to fix
v0.0.23 Internal client representation changed The cluster has to be re-build from scratch

---

[ 27.11.2023 ] v0.0.22

Description

Hard-coded use case 4 app definition and safe deposit schema have been added to allow integration tests.

[ 21.11.2023 ] v0.0.21

Description

In order to allow monitoring and observation of cluster activities and to visualize the gathered data, Opentelemetry and Grafana were introduced.

The configuration of keycloak was refactored and a new endpoint for the registration of safe deposit box users was implemented. Quarkus was updated to version 3.5.0.

Added features

  • Grafana
  • Opentelemetry
  • Quarkus update to 3.5.0
  • Endpoint to register safe deposit box users

[ 07.11.2023 ] v0.0.20

Description

Safe deposit boxes are now fully integrated to offer data provider a persistent secure storage of data which can be accessed by a EuroDaT transaction.

The RabbitMQ PoC has been extended to a full integration into EuroDaT and can now be used by data apps to send messages to the data providers.

To further improve the security of our system, we now use short-lived credentials in our cluster. Using our Docker container vulnerability scanner we have updated the affected images.

To speed up local development, we added support for network policies using Calico in our Kind/tilt setup.

Added features

  • Short-lived credentials for cluster
  • Safe deposit boxes
  • Full EuroDaT integration of RabbitMQ
  • Support of k8s network policies using Calico in KinD
  • Added ArgoWorkflow templates for UC2 & safeAML (currently hardcoded)

[ 24.10.2023 ] v0.0.19

Description

The database-service, transaction-service, and credential-service are now deployed separately, i.e., they are no longer deployed in the same pod as the controller.
To further improve the security of our system, we implemented a pipeline job to scan Docker Containers for vulnerabilities and changed the kube-config files for our Azure clusters to use kubelogin.
First steps to introduce a back-messaging service were taken. RabbitMQ was deployed.

Added features

  • database-service, transaction-service, and credential-service as separate pods.
  • Trivy pipeline job to scan Docker Images.
  • kubelogin for Azure kube-config files.
  • RabbitMQ

[ 09.10.2023 ] v0.0.18

Description

Continuing the transition to a microservice architecture, the database-service with its client was created and the transaction-service was extracted from the controller. The new service also has the capabilities to provision safe deposit boxes.

Added features

  • Endpoints to provision safe deposit boxes
  • QA environment on GCP
  • Tests to verify row-level security
  • Force TLSv1.3 for connections to Vault

[ 25.09.2023 ] v0.0.17

Description

Current release focusing on security and stability enhancements, besides our internal release pipeline (further) development. In the documentation the Guide sections are revised and if needed corrected. This release also includes some smaller bug fixes.

Added features

  • Upgraded Quarkus to 3.x as part of security improvements.
  • Testing strategy implemented for GitOps deployments (ArgoCD).
  • Provisioning of QA deployment stage and promotion between stages.
  • Documentation improvements in “Guides” section.
  • Fixed sometimes occurring Transaction Vault startup failure.
  • Fix cert-manager deployment for ArgoCD.

[ 12.09.2023 ] v0.0.16

Description

For the transition to a microservice architecture, the transaction-service-client and the credential-service-client were created.

To increase security, multiple measures were taken:

  • The ingresses pointing to the client-controllers' API were removed.
  • Rate limiting for the platform and IAM APIs was introduced.
  • The default deployment of the platform now uses strong auto-generated passwords.
  • Authentication is now enforced at all platform API endpoints.
  • The platform ingress does not point to the user resource anymore and the resource was removed from our Swagger UI.

The official documentation received an overhaul.
For testing, the uc3-mock-workflow and the safeAML-mock-algorithm-execution container images were created and are now part of the released images.
RabbitMQ is now part of the deployment and the published packages.
New e2e tests were implemented which check if provided credentials are valid and if created databases are deleted after a transaction. Row-based security can be enabled in the output schema. This is achieved by assigning a security value to each client and defining a security column.

Added features

  • Authentication of API endpoints.
  • Auto-generated passwords.
  • API rate limiting.
  • The RabbitMQ Helm chart.
  • The e2e tests: testGetCredentials & testDatabaseDeletionAfterTransactionEnd.
  • Row-based security can be enabled in the output schema of an app.

Deleted features

  • Ingress to the client-controllers' API.
  • Public access to the user resource.

[ 28.08.2023 ] v0.0.15

Description

The roles and credentials for the temporary PostgreSQL database are now generated by Hashicorp Vault. Due to Azure Key Vault offering the required functionality with fewer obstacles in terms of production hardening, it supersedes the use of Vault in the management cluster. Furthermore, a readiness/health check for all components is added and integrated into the pipeline to check if they work even if they are not used in an e2e test.

Changed features

  • Temporary PostgreSQL database with credential generation by Hashicorp Vault which is initiated upon transaction start and automatically ceases after completion
  • External secrets now utilize Azure Key Vault as the backend, replacing Vault

Added features

  • Implemented readiness/health checks for all components and integrated them into the pipeline

[ 15.08.2023 ] v0.0.14

Description

The development and deployment of the EuroDaT can now be done on GCP. In this sprint a new mock container for the algorithm execution of the safeAML workflow was implemented with an Argo workflow template which is triggered in the start workflow endpoint. Furthermore, the ice cream app was fixed and can be used for demonstrating the eurodat principles. The Hashicorp Vault is finalized and can be used in production, a diagram for the interaction between the individual components of EuroDaT and the Hashicorp Vault was added.

Changed features

  • Ice cream app can be used again after fixing a bug
  • Hashicorp Vault can now be used in production

Added features

  • Deployment on GCP
  • Mock Container for algorithm execution in the safeAML workflow

[ 02.08.2023 ] v0.0.13

Description

This update brings important changes to our trust chains and certificate passing rules, resulting in a more simplified and secure process. To enhance efficiency and security we now solely trust two CAs instead of trusting all leaf certificates one by one and external secrets are used to fetch secrets from the vault. Furthermore, to increase the readability of docs.eurodat.org, navigation is improved, broken links are fixed, and internal remarks are removed.

Changed features

  • Enhanced security measures Enhanced security measures using: self-signed CAs and Let's Encrypt.
  • Navigation on docs.eurodat.org is updated and its readability is increased.

Added features

  • External secrets

[ 20.07.2023 ] v0.0.12

Description

As part of the code clean-up regarding legacy components, all EDC extensions are removed. The documentation is therefore updated and also generally refactored to simplify orientation and usability. The usage of Talisman is extended.

Changed features

  • Talisman was added as a linter job.
  • Improved configuration for macOS
  • Documentation was updated on several topics (CI/CD, removal of broker/EDC)

Added features

  • Vault UI
  • Definition of safeAML schema definitions

Deleted features

  • all EDC extensions

[ 04.07.2023 ] v0.0.11

Description

The legacy components operator and privileged flash vaults are removed. Pull-asset and extract-asset are substituted by uploads/downloads to a per-transaction database. Hashicorp Vault grants authenticated and authorized clients access to the database. The folder structure and the docs in the project are adapted accordingly.

Changed features

  • New docs will only be available with each release.
  • Controller is now either broker-controller or client-controller.

Added features

  • A transaction database on the PostgreSQL server that is available for the duration of a transaction which is started by an app execution. Input, intermediate and output schemata are created. Roles and credentials are generated. Workflows can be started end ended. The connection is via JDBC.
  • Public repository usage for Helm Charts without tokens

Deleted features

  • Operator
  • Privileged flash vaults (pull-asset, extract-asset)

[ 20.06.2023 ] v0.0.10

Description

We improved the functionality of the dev environment with a more comprehensive Makefile, more test results during merge requests and checks to avoid committing secrets to the repo. We created a client-side application.

Changed features

  • Makefile is restructured and improved. It installs the complete local environment for Ubuntu with all its dependencies if needed.
  • Talisman is added to the pre-commit hook script to check whether secrets are committed.
  • Cert-Manager is improved.

Added features

  • SonarCloud test results visible in merge requests
  • Client-side application that provides simple endpoints, triggers workflows and allows pre-configuring all instance-specific details
  • New IaC repository with Terraform files
  • Policy store which persists data access policies

[ 06.06.2023 ] v0.0.9

Description

UC3 app translated into Argo Workflow Template. Gradle, ktlint and some other packages were updated. Argo CD setup was improved. PoC: Dataflow in transaction and PostgreSQL Credential Management. New API endpoint was implemented to start or end a data transaction belonging to the application

Added features

  • UC3 app is now a Workflow Template with artifact repository
  • Dev deployment can now be done with Argo CD
  • Image and package versions updated
  • Endpoint to start or end a data transaction introduced

[ 23.05.2023 ] v0.0.8

Description

A new Quarkus server and PostgreSQL dev container were set up in the project using Liquibase; HTTP response headers added.

Added features

  • HTTP response headers for security
  • A new Quarkus server and PostgreSQL dev container

[ 12.05.2023 ] v0.0.7

Description

Argo Workflows has been extended with a reactive java client and a workflow template for our test flow with external input was created

Changed features

  • EDC updated to milestone-8.
  • Removal of Postman related resources.
  • Updated documentation.

Added features

  • Reactive java client for Argo Workflows.
  • Workflow template for test flow with external input.
  • Tutorial for Swagger UI.

[ 26.04.2023 ] v0.0.6

Description

Argo Workflows has been integrated into the EuroDaT deployment. It will replace the operator as a Kubernetes orchestrator in future releases. Also, the ice cream demonstrator mock was finalized.

Changed features

  • Kubernetes for Azure (AKS) was changed from version 1.23.12 to 1.25.6

Added features

  • Argo Workflow has been added to the deployment of the main cluster.
  • A client for communicating with the Argo Workflow server is now available in the EuroDaT controller.
  • A PostgreSQL instance has been set up to archive workflows.
  • Added the option to exclude services in Tilt to save resources.

[ 13.04.2023 ] v0.0.5

Description

Authentication using Keycloak has been added to the ice cream selector Angular application (used for showcasing EuroDaT's fundamental concepts).

Added features

  • Keycloak integration has been added to the ice cream selector app.

[ 30.03.2023 ] v0.0.4

Description

For an interactive showcase of EuroDaT's fundamental concepts, such as data sovereignty, two Angular applications and a python flash vault have been added.

The provisioning of flash vaults and asset registration have been moved from the EDC-based broker to the controller. EDC extensions which are no longer required have been removed.

Added features

  • Endpoints in controller for:
    • asset registration
    • triggering app execution
  • Performance improvements for Tilt by separating resources and image builds
  • Added a documentation folder to help in the UC onboarding process along with a quick-start guide for the connector-subchart with an example configuration.

[ 14.03.2023 ] v0.0.3

Description

A Helm subchart has been added and published that allows one to deploy a plain EDC connector with pre-registered technical assets, pre-configured TLS communication and IAM-service. Additional optional features include persistence for EDC resources and logs, an API controller and a separable data-plane.

Added features

  • Helm subcharts for client and broker EDC-connectors

[ 27.02.2023 ] v0.0.2

Description

Authorization for controllers and Swagger UI addition.

Added features

  • /assets/{assetId} endpoint in controller was added
  • Authentication for all controller endpoints is enabled
  • Requests for asset operations require authorization
  • Custom-data-plane was added to allow requests for asset data
  • Openapi spec and Swagger UI was added
  • Added reloader to deployment for automatic reloading on config and secret changes

[ 16.02.2023 ] v0.0.1 Initial release of the EuroDaT Project

Description

The EuroDaT Platform is the software application operated by the European Data Trustee EuroDaT. It provides a trusted, safe, compliant and transparent platform to exchange and evaluate data. The actual version offers a user the possibility to store and access registered data. Preliminary features also include a simple contract negotiation flow and a basic authentication flow.