Developing a Connector
A Connector is a Microservice that integrates a Data Service (like Owncloud or Zenodo) into RDS and is located in Layer 1. It has to register to the Token Storage for it to be able to provide a authentication process within RDS. The Data Service needs to be able to perform authentication via oauth2 and you must register an application with the storage provider to get API access. The Token Storage requires all Oauth2 workflow information (Client ID, Client Secret, authorize url, etc.) of the data storage. Registration is done via the service
endpoint of the Token Storage.
The first action of your connector should be to login to the service it connects to - the login should certainly happen before the connector starts it's own API.
Your connector corresponds to one or both of two different types of service connectors, each one having their own interface in form of a OpenAPIv3 specification to implement. The distinction between service connector types is relevant when it comes to internal communication with other microservices.
The different connector types are file storage and metadata storage:
Interface | Implements | Example Connector | Description |
---|---|---|---|
File Storage | fileStorage | Owncloud | Used to communicate with EFSS backend. Mostly to get files. |
Metadata Storage | metadata | Zenodo | Used to communicate with repositories and archiving services. Mostly to push files to. |
Registering with the Token Storage
The following is an example function of how to register your service connector with RDS' Token Storage, written in Python.
def register_service(servicename: str, authorize_url: str, refresh_url: str, client_id: str, client_secret: str):
tokenStorage = os.getenv("CENTRAL_SERVICE_TOKEN_STORAGE")
if tokenStorage is None:
return False
data = {
"servicename": servicename,
"authorize_url": authorize_url,
"refresh_url": refresh_url,
"client_id": client_id,
"client_secret": client_secret,
"implements": ["fileStorage", "metadata"]
}
headers = {"Content-Type": "application/json"}
response = requests.post(
f"{tokenStorage}/service", json=data, headers=headers)
if response.status_code is not 200:
raise Exception(
"Cannot find and register Token Storage, msg:\n{}".format(response.text)
response = response.json()
if response["success"]:
logger.info(
f "Registering {servicename} in token storage was successful.")
return True
logger.error(
f "There was an error while registering {servicename} to token storage.\nJSON: {response}")
return False
Note that the data dict has an implements
-key, which contains a list that holds the name(s) of the interface you choose to implement. You have to set it accordingly.
Connexion-Plus
We strongly recommend using Connexion-Plus to make working with Flask and Python easier.
Containerisation
Place your connector inside of a docker container, so it can be used in a kubernetes environment. All your configuration should be available through environment variables or a single configuration file, placed in a single, very specific file location.