User Tools

Site Tools


appds:arch

APPDS Architecture

Main ideas that are embedded in the architecture:
1) lack of intervention in local storage, adapters are used for data access;
2) user requests are processed on a dedicated server based on metadata only.

  • S1, S2, … - Local data storage. We consider the local storage as a file storage.
  • In1, In2 - Data sources. There are two types of sources: primary data sources like experimental setups (In1) and secondary data sources (In2). An example of secondary data source may be program which calibrate the primary data.
  • MDD1, MDD2, … - Metadata description. It is assumed that this is a description of the structure of binary files (Kaitai) with an indication of what is metadata.
  • E1, E2, … - Metadata extractors. These are services that ensure the retrieval of meta-information and the registration of incoming data in the metadata database.
  • A1, A2, … - Adapters. Adapters provide unified API to access to the data on local storage. We propose to use CERNVM-FS as a standard adapter.
  • MDС - Metadata catalogue. This is a service which support two main function. First - register collected metadata, adn second - process the user requests for data. This is the single place where location of necessary data define taking into account user request.
  • AS - Aggregation service. This service accept user requests and aggregate corresponding data into new collection taking into account the request. The request may contain either file-level criteria (for example session date) or event-level criteria (for example event energy).
  • TPL - Template library. The collection of the programs is used for aggregating data.
  • UI - User interface - the Web interface for user access.
  • Application service - the external service which provide the access to external (computing) resources.
appds/arch.txt · Last modified: 04/04/2019 08:01 by kryukov