Skip to main content

Developing a Connector

A Connector is a Microservice that integrates a Data Service (like Owncloud or Zenodo) into RDS and is located in layer 1. It has to register to the Token Storage for it to be able to provide a authentication process within RDS. The Data Service needs to be able to perform authentication via oauth2 and you must be register an application with the storage provider to get API access. The Token Storage requires all Oauth2 workflow information (Client ID, Client Secret, authorize url, etc.) of the data storage. Registration is done via the service endpoint of the Token Storage.

sequenceDiagram participant PS as Port Service participant TS as Token Storage PS->>TS: Registration

The first action of your connector should be to login to the service it connects to - the login should certainly happen before the connector starts it's own API.
Your connector corresponds to one or both of two different types of service connectors, each one having their own interface in form of a OpenAPIv3 specification to implement. The distinction between service connector types is relevant when it comes to internal communication with other microservices.

The different connector types are file storage and metadata storage:

InterfaceImplementsExample ConnectorDescription
File StoragefileStorageOwncloudUsed to communicate with EFSS backend. Mostly to get files.
Metadata StoragemetadataZenodoUsed to communicate with repositories and archiving services. Mostly to push files to.

Registering with the Token Storage

The following is an example function of how to register your service connector with RDS' Token Storage, written in Python.

def register_service(servicename: str, authorize_url: str, refresh_url: str, client_id: str, client_secret: str):
tokenStorage = os.getenv("CENTRAL_SERVICE_TOKEN_STORAGE")
if tokenStorage is None:
return False

data = {
"servicename": servicename,
"authorize_url": authorize_url,
"refresh_url": refresh_url,
"client_id": client_id,
"client_secret": client_secret,
"implements": ["fileStorage", "metadata"]
}
headers = {"Content-Type": "application/json"}

response = requests.post(
f"{tokenStorage}/service", json=data, headers=headers)

if response.status_code is not 200:
raise Exception(
"Cannot find and register Token Storage, msg:\n{}".format(response.text)

response = response.json()
if response["success"]:
logger.info(
f "Registering {servicename} in token storage was successful.")
return True

logger.error(
f "There was an error while registering {servicename} to token storage.\nJSON: {response}")

return False

Note that the data dict has an implements-key, which contains a list that holds the name(s) of the interface you choose to implement. You have to set it accordingly.

Connexion-Plus

We strongly recommend using Connexion-Plus to make working with Flask and Python easier.

Containerisation

Place your connector inseide of a docker container, so it can be used in a kubernetes environment. All your configuration should be able through environment variables or a single file, which is placed in a single, very specific file location.