Release Notes

Here we document all major changes in a release, incl. added and deprecated functionalities. In particular, we provide a comprehensive list of system-breaking changes that could affect end users.

Breaking Changes

Version number	Change	How to fix

[08.11.2024] v0.0.45

Description

This release introduces updates for platform egress and backend connectivity. Furthermore, it focuses on extending the API, along with improvements in dependency management and efficiency within the CI/CD pipeline.

Added features

Enabled egress, allowing specific services to operate with external access.
Data Management API: New REST endpoints to upload and retrieve participant data.
REST API for Messaging: Clients can poll messages through REST, with endpoints for message posting and retrieval.
Refined Renovate configuration and automated lock file maintenance.
Various image dependencies are updated.
Various bugs are fixed.

Removed Features

RabbitMQ: Removed as part of transitioning messaging functionality to the REST API.

[15.10.2024] v0.0.44

Description

This release focuses on extending the API, optimizing the load balancing, updating some image dependencies, and fixing various bugs.

Added features

QA+ deployment and promotion is removed from the platform.
Error in e2e port forwarding in the registration of test users is fixed.
GET messages endpoint is added.
Load balancer with external static IP is enabled.
Various image dependencies are updated to the latest versions.
Various bugs are fixed.

[08.10.2024] v0.0.43

Description

This release enables external GCP load balancing and exposes a client registration API. Some dependencies have been updated.

Added features

Platform can be deployed with an external network pass-through loadbalancer in GCP.
A client registration API is exposed.

[01.10.2024] v0.0.42

Description

This release focuses on extending the frontend functionalities, updating some image dependencies, and fixing various bugs.

Added features

Safedeposit roles are moved to Safedeposit lifecycle.
Asking for confirmation of contract is added.
Signer status is added to the dashboard.
Cypress-e2e test is extended.
Various image dependencies are updated to the latest versions.
Various bugs are fixed.

[11.09.2024] v0.0.41

Description

This release focuses on improving the contract management and onboarding, as well as various bug fixes.

Added features

Contract status is updated upon page reload.
Timestamp of last contract status update displayed in UI.
Participants persistantly stored for contracts.
Update of opentelemetry and bitnami versions.
Various bugs are fixed.

[04.09.2024] v0.0.40

Description

This release focuses on workstation improvements, including ArgoCD access, image upload workflow, and various bug fixes.

Added features

Instructions are added to access ArgoCD instances from client machine instances via workstation.
Workstation setup documentations are updated.
The contract status in frontend can now be retrieved.
Participant and company are now stored in database.
Various image dependencies are updated to the latest versions.
Various bugs are fixed.

[19.08.2024] v0.0.39

Description

This release focuses on platform improvements, including frontend styling, documentation updates, and bug fixes. Furthermore, some image dependencies have been updated to the latest versions.

Added features

Callback from paperless.io after submission is implemented.
Styling for mobile devices in frontend is finished.
Onboarding documentations are updated.
Various image dependencies are updated to the latest versions.
Various bugs are fixed.

[07.08.2024] v0.0.38

Description

This release adds a contract management service with a post endpoint to start the paperless workflow and contains several bug fixes as well as updated dependencies.

Added features

The paperless workflow can be started from a new endpoint.
Restyling of mobile status progress bar.
Bug fixes.

[26.07.2024] v0.0.37

Description

This release focuses on unifying the external client interface and implementing SSO for Keycloak clients in the frontend. Additionally, it includes some improvements and bug fixes, and the removal of the external cluster.

Added features

The external client interface is unified.
Error handling for role creation is improved.
SSO is implemented for Keycloak clients in the frontend.
Various bugs are fixed.

[09.07.2024] v0.0.36

Description

This release focuses mainly on some bugfixes, improvements to the development workflow and integrating Keycloak as an identity provider to the TYPO3 frontend.

Added features

New e2e tests are added for the contract service.
The config job now receives a random suffix. Therefore, the sync does not fail updating the immutable job spec.
The error handling in credential resource of transaction database is improved.
Some bugs are fixed.

[26.06.2024] v0.0.35

Description

This release integrates paperless.io to enable the digital signing of contracts and implements a first process.

Added features

Workflows can now be queried at a new API endpoint.
Response codes and messages of safe deposit endpoints reflect server and client errors more precisely.
Our backend is integrated with paperless.io.
The Company Master Data workflow can be triggered in the EuroDaT Dashboard.

[23.05.2024] v0.0.34

Description

This release introduces a true GitOps Deployment and allows dynamic registration of apps at EuroDaT.

Added features

The desired state of the deployment is now entirely defined in git.
Apps can be registered dynamically if authorized.
The nightly build is now a full nightly release.
Database queries and updates received hardening.
Finalisation of frontend development and testing setup.

[29.04.2024] v0.0.33

Description

This release focuses on the frontend of the EuroDaT platform.

Added features

Transfer of our angular components to angular widgets
Cypress is now used for frontend e2e testing
Registration, login, and authentication for EuroDaT frontend
External UC image testing was moved from dev to integration envoronment
External Secrets Operator was implemented

[17.04.2024] v0.0.32

Description

This release enhances Eurodat web frontend and service documentation.

Added features

Keycloak for EuroDat web frontend has been configured
Hardening of the web container has been implemented
Contract-service has been implemented
Automatic validation of workflows and workflow templates (names, format, etc.) during dynamic app registration has been implemented
Copy-and-Paste-Logic for MiNaBo app has been added
Documentation of services has been updated
Clusters are now deployed with an own artifact registry for UC images

[ 04.04.2024 ] v0.0.31

Description

This release enhances the dynamic app registration process and hardcodes a new Minabo app.

Added features

Minabo app now has a Hello World workflow
Dynamic app registration now registers a workflow template when registering an app
app-of-apps helm chart is now part of the trustee-plattform repository
Enhanced app-of-apps helm charts to support GCP private cluster deployment

[ 20.03.2024 ] v0.0.30

Description

This release provides bugfixes, improvements to the development workflow and maintainability. In addition, we started working on enabling service providers to register new apps. Further, we have started integrating Keycloak as an identity provider for the user journey login process.

Added features

Added creation of database roles when registering a new app
Deduplicated cloud provider versions of app-of-apps helm chart
Deduplicated stages of app-of-apps helm chart
Integrated Keycloak in Angular login component

[ 08.03.2024 ] v0.0.29

Description

This release now provides an API to dynamically register new data apps which does not require code changes within the repository or an updated release to get a new schema (safe deposit box / transactional databases) and app definition (Argo Workflow Template) into EuroDaT workflow engine. We further continued to work on the onboarding frontend which now integrates into a dummy EuroDaT homepage.

Added features

Added login/sign-up options (dummy only) to the EuroDaT homepage (dummy only) for participant onboarding
Added API to dynamically register data apps

[ 20.02.2024 ] v0.0.28

Description

This release focuses mainly on bug fixes concerning deployment stability and further code refactoring to a microservice architecture. To take care of the participant onboarding we have started to work on the EuroDaT platform web frontend.

Added features

We have started to remove obsolete broker appearances
The sceletal structure for participant onboarding web frontend has been created

[ 06.02.2024 ] v0.0.27

Description

This release includes new features which improve the permission management and updates for more security by using an image digest.

Added features

We increased security by using an image digest to refer to images to ensure that the image that is pulled or pushed is the exact one we intend to push or pull.
The AppRepository is split into its own microservice for a cleaner architecture.
To ensure proper permission management for various apps, we added an explicit specification for the participants of every transaction as a prerequisite to check permissions.
Fixed bugs for the nightly pipeline.

[ 24.01.2024 ] v0.0.26

Description

A small release that adds backmessaging service features and some bug fixes

Added features

Transaction participants are now informed by a backmessage if a transaction or workflow is started
The image registry for the client controller is set properly
ArgoWorkflows mock pods are now cleaned up after completion without errors
Fixed a bug where the backmessaging service helm chart was not published in a release

[ 12.01.2024 ] v0.0.25

Description

A small release with refactorings and stabilisation changes

Added features

safeAML and UC4 Safe Deposit Box Schema Update
UC2 workflow template fix
The RabbitMQ user from the container can now access the queues of all transaction participants
Simultanious calls in user resource are now handled correctly

[ 19.12.2023 ] v0.0.24

Description

A small release with refactorings and stabilisation changes

Added features

UC4 Safe Deposit Box Schema Update
Release pipeline is a full pipeline now
Microservice use Flyway to manage their schemas
Automated test checks that all EuroDaT endpoints require authentication

Breaking Changes

Version number	Change	How to fix
v0.0.24	Postgres instances merged	Run reset_db.sh

[ 12.12.2023 ] v0.0.23

Description

ArgoCD is used for all non-local deployments of EuroDaT to a cluster, replacing the previous Helm CLI deployment to development environments. EuroDaT now supports dynamic registration of clients together with their client selector, replacing the previous hardcoding. Tempo and Prometheus were introduced to enhance monitoring of the cluster. EuroDaT onboarding documentation has received a great overhaul. Kubernetes was updated to 1.27.7. OpenCost has been removed from EuroDaT. EuroDaT now provides a integration environment with the official latest release.

Added features

Dynamic client registration

Breaking Changes

Version number	Change	How to fix
v0.0.23	Internal client representation changed	The cluster has to be re-build from scratch

---

[ 27.11.2023 ] v0.0.22

Description

Hard-coded use case 4 app definition and safe deposit schema have been added to allow integration tests.

[ 21.11.2023 ] v0.0.21

Description

In order to allow monitoring and observation of cluster activities and to visualize the gathered data, Opentelemetry and Grafana were introduced.

The configuration of keycloak was refactored and a new endpoint for the registration of safe deposit box users was implemented. Quarkus was updated to version 3.5.0.

Added features

Grafana
Opentelemetry
Quarkus update to 3.5.0
Endpoint to register safe deposit box users

[ 07.11.2023 ] v0.0.20

Description

Safe deposit boxes are now fully integrated to offer data provider a persistent secure storage of data which can be accessed by a EuroDaT transaction.

The RabbitMQ PoC has been extended to a full integration into EuroDaT and can now be used by data apps to send messages to the data providers.

To further improve the security of our system, we now use short-lived credentials in our cluster. Using our Docker container vulnerability scanner we have updated the affected images.

To speed up local development, we added support for network policies using Calico in our Kind/tilt setup.

Added features

Short-lived credentials for cluster
Safe deposit boxes
Full EuroDaT integration of RabbitMQ
Support of k8s network policies using Calico in KinD
Added ArgoWorkflow templates for UC2 & safeAML (currently hardcoded)

[ 24.10.2023 ] v0.0.19

Description

The database-service, transaction-service, and credential-service are now deployed separately, i.e., they are no longer deployed in the same pod as the controller.
To further improve the security of our system, we implemented a pipeline job to scan Docker Containers for vulnerabilities and changed the kube-config files for our Azure clusters to use kubelogin.
First steps to introduce a back-messaging service were taken. RabbitMQ was deployed.

Added features

database-service, transaction-service, and credential-service as separate pods.
Trivy pipeline job to scan Docker Images.
kubelogin for Azure kube-config files.
RabbitMQ

[ 09.10.2023 ] v0.0.18

Description

Continuing the transition to a microservice architecture, the database-service with its client was created and the transaction-service was extracted from the controller. The new service also has the capabilities to provision safe deposit boxes.

Added features

Endpoints to provision safe deposit boxes
QA environment on GCP
Tests to verify row-level security
Force TLSv1.3 for connections to Vault

[ 25.09.2023 ] v0.0.17

Description

Current release focusing on security and stability enhancements, besides our internal release pipeline (further) development. In the documentation the Guide sections are revised and if needed corrected. This release also includes some smaller bug fixes.

Added features

Upgraded Quarkus to 3.x as part of security improvements.
Testing strategy implemented for GitOps deployments (ArgoCD).
Provisioning of QA deployment stage and promotion between stages.
Documentation improvements in “Guides” section.
Fixed sometimes occurring Transaction Vault startup failure.
Fix cert-manager deployment for ArgoCD.

[ 12.09.2023 ] v0.0.16

Description

For the transition to a microservice architecture, the transaction-service-client and the credential-service-client were created.

To increase security, multiple measures were taken:

The ingresses pointing to the client-controllers' API were removed.
Rate limiting for the platform and IAM APIs was introduced.
The default deployment of the platform now uses strong auto-generated passwords.
Authentication is now enforced at all platform API endpoints.
The platform ingress does not point to the user resource anymore and the resource was removed from our Swagger UI.

The official documentation received an overhaul.
For testing, the uc3-mock-workflow and the safeAML-mock-algorithm-execution container images were created and are now part of the released images.
RabbitMQ is now part of the deployment and the published packages.
New e2e tests were implemented which check if provided credentials are valid and if created databases are deleted after a transaction. Row-based security can be enabled in the output schema. This is achieved by assigning a security value to each client and defining a security column.

Added features

Authentication of API endpoints.
Auto-generated passwords.
API rate limiting.
The RabbitMQ Helm chart.
The e2e tests: testGetCredentials & testDatabaseDeletionAfterTransactionEnd.
Row-based security can be enabled in the output schema of an app.

Deleted features

Ingress to the client-controllers' API.
Public access to the user resource.

[ 28.08.2023 ] v0.0.15

Description

The roles and credentials for the temporary PostgreSQL database are now generated by Hashicorp Vault. Due to Azure Key Vault offering the required functionality with fewer obstacles in terms of production hardening, it supersedes the use of Vault in the management cluster. Furthermore, a readiness/health check for all components is added and integrated into the pipeline to check if they work even if they are not used in an e2e test.

Changed features

Temporary PostgreSQL database with credential generation by Hashicorp Vault which is initiated upon transaction start and automatically ceases after completion
External secrets now utilize Azure Key Vault as the backend, replacing Vault

Added features

Implemented readiness/health checks for all components and integrated them into the pipeline

[ 15.08.2023 ] v0.0.14

Description

The development and deployment of the EuroDaT can now be done on GCP. In this sprint a new mock container for the algorithm execution of the safeAML workflow was implemented with an Argo workflow template which is triggered in the start workflow endpoint. Furthermore, the ice cream app was fixed and can be used for demonstrating the eurodat principles. The Hashicorp Vault is finalized and can be used in production, a diagram for the interaction between the individual components of EuroDaT and the Hashicorp Vault was added.

Changed features

Ice cream app can be used again after fixing a bug
Hashicorp Vault can now be used in production

Added features

Deployment on GCP
Mock Container for algorithm execution in the safeAML workflow

[ 02.08.2023 ] v0.0.13

Description

This update brings important changes to our trust chains and certificate passing rules, resulting in a more simplified and secure process. To enhance efficiency and security we now solely trust two CAs instead of trusting all leaf certificates one by one and external secrets are used to fetch secrets from the vault. Furthermore, to increase the readability of docs.eurodat.org, navigation is improved, broken links are fixed, and internal remarks are removed.

Changed features

Enhanced security measures Enhanced security measures using: self-signed CAs and Let's Encrypt.
Navigation on docs.eurodat.org is updated and its readability is increased.

Added features

External secrets

[ 20.07.2023 ] v0.0.12

Description

As part of the code clean-up regarding legacy components, all EDC extensions are removed. The documentation is therefore updated and also generally refactored to simplify orientation and usability. The usage of Talisman is extended.

Changed features

Talisman was added as a linter job.
Improved configuration for macOS
Documentation was updated on several topics (CI/CD, removal of broker/EDC)

Added features

Vault UI
Definition of safeAML schema definitions

Deleted features

all EDC extensions

[ 04.07.2023 ] v0.0.11

Description

The legacy components operator and privileged flash vaults are removed. Pull-asset and extract-asset are substituted by uploads/downloads to a per-transaction database. Hashicorp Vault grants authenticated and authorized clients access to the database. The folder structure and the docs in the project are adapted accordingly.

Changed features

New docs will only be available with each release.
Controller is now either broker-controller or client-controller.

Added features

A transaction database on the PostgreSQL server that is available for the duration of a transaction which is started by an app execution. Input, intermediate and output schemata are created. Roles and credentials are generated. Workflows can be started end ended. The connection is via JDBC.
Public repository usage for Helm Charts without tokens

Deleted features

Operator
Privileged flash vaults (pull-asset, extract-asset)

[ 20.06.2023 ] v0.0.10

Description

We improved the functionality of the dev environment with a more comprehensive Makefile, more test results during merge requests and checks to avoid committing secrets to the repo. We created a client-side application.

Changed features

Makefile is restructured and improved. It installs the complete local environment for Ubuntu with all its dependencies if needed.
Talisman is added to the pre-commit hook script to check whether secrets are committed.
Cert-Manager is improved.

Added features

SonarCloud test results visible in merge requests
Client-side application that provides simple endpoints, triggers workflows and allows pre-configuring all instance-specific details
New IaC repository with Terraform files
Policy store which persists data access policies

[ 06.06.2023 ] v0.0.9

Description

UC3 app translated into Argo Workflow Template. Gradle, ktlint and some other packages were updated. Argo CD setup was improved. PoC: Dataflow in transaction and PostgreSQL Credential Management. New API endpoint was implemented to start or end a data transaction belonging to the application

Added features

UC3 app is now a Workflow Template with artifact repository
Dev deployment can now be done with Argo CD
Image and package versions updated
Endpoint to start or end a data transaction introduced

[ 23.05.2023 ] v0.0.8

Description

A new Quarkus server and PostgreSQL dev container were set up in the project using Liquibase; HTTP response headers added.

Added features

HTTP response headers for security
A new Quarkus server and PostgreSQL dev container

[ 12.05.2023 ] v0.0.7

Description

Argo Workflows has been extended with a reactive java client and a workflow template for our test flow with external input was created

Changed features

EDC updated to milestone-8.
Removal of Postman related resources.
Updated documentation.

Added features

Reactive java client for Argo Workflows.
Workflow template for test flow with external input.
Tutorial for Swagger UI.

[ 26.04.2023 ] v0.0.6

Description

Argo Workflows has been integrated into the EuroDaT deployment. It will replace the operator as a Kubernetes orchestrator in future releases. Also, the ice cream demonstrator mock was finalized.

Changed features

Kubernetes for Azure (AKS) was changed from version 1.23.12 to 1.25.6

Added features

Argo Workflow has been added to the deployment of the main cluster.
A client for communicating with the Argo Workflow server is now available in the EuroDaT controller.
A PostgreSQL instance has been set up to archive workflows.
Added the option to exclude services in Tilt to save resources.

[ 13.04.2023 ] v0.0.5

Description

Authentication using Keycloak has been added to the ice cream selector Angular application (used for showcasing EuroDaT's fundamental concepts).

Added features

Keycloak integration has been added to the ice cream selector app.

[ 30.03.2023 ] v0.0.4

Description

For an interactive showcase of EuroDaT's fundamental concepts, such as data sovereignty, two Angular applications and a python flash vault have been added.

The provisioning of flash vaults and asset registration have been moved from the EDC-based broker to the controller. EDC extensions which are no longer required have been removed.

Added features

Endpoints in controller for:
- asset registration
- triggering app execution
Performance improvements for Tilt by separating resources and image builds
Added a documentation folder to help in the UC onboarding process along with a quick-start guide for the connector-subchart with an example configuration.

[ 14.03.2023 ] v0.0.3

Description

A Helm subchart has been added and published that allows one to deploy a plain EDC connector with pre-registered technical assets, pre-configured TLS communication and IAM-service. Additional optional features include persistence for EDC resources and logs, an API controller and a separable data-plane.

Added features

Helm subcharts for client and broker EDC-connectors

[ 27.02.2023 ] v0.0.2

Description

Authorization for controllers and Swagger UI addition.

Added features

/assets/{assetId} endpoint in controller was added
Authentication for all controller endpoints is enabled
Requests for asset operations require authorization
Custom-data-plane was added to allow requests for asset data
Openapi spec and Swagger UI was added
Added reloader to deployment for automatic reloading on config and secret changes

[ 16.02.2023 ] v0.0.1 Initial release of the EuroDaT Project

Description

The EuroDaT Platform is the software application operated by the European Data Trustee EuroDaT. It provides a trusted, safe, compliant and transparent platform to exchange and evaluate data. The actual version offers a user the possibility to store and access registered data. Preliminary features also include a simple contract negotiation flow and a basic authentication flow.