dfsfsdf

Orchard Specification

2018-02-05

Authorship and Legal

Orchard is intended to be a scalable service orchestration layer for Service Oriented Architecture.

It is wholly designed and implemented by Chris Punches in 2018.  It is owned by this project’s parent group, IRCTHULU and consequently SILO GROUP, LTD. and Chris Punches, and all rights are reserved.

You may not create or implement designs based on it without my explicit permission in each case.

Abstract

Orchard is the developing result of a project called IRCTHULU.   It was originally termed “T-ORCH” but was later renamed after its central component, called “Orchard”.

Orchard allows you to centrally control and administrate, and orchestrate, services and microservices across environments or within the same environment.

It is intended for complex application layer orchestration as a command and control system triggering actions within the services that are aware of it.

The diagram below outlines component interoperation at an abstract level:

Components

  • The controller, which will be called bonebox.
  • The API, which will be called orchard.
  • The MQ, which will always be just MQ.
  • The consumer, which will be called harvester.
  • The database, which will always be just DB.
  • The service, which is what this whole thing revolves around and isn’t part of the solution really, but needs to be aware of how it should interact, presumably through shared library calls.  For simplicity we’ll just call this service but it’s technically a client to orchard.

Bonebox Controller

The bonebox controller registers a request or cancels a request.  It also can check on the state of requests.  It is the point at which bonebox GUIDs are registered.  It is able to obtain all available service GUIDs and request GUIDs.

Orchard API

The Orchard RESTful API reports on state for requests, creates, updates, reports, and deletes requests from the controller, which it relays to the MQ for buffering.  It polls request state directly from the DB,  provides a way for services to check for new requests, as well as sending request state updates from the service to the MQ.  It is the point at which request GUIDs and bonebox controller GUIDs are created.  It is the point at which GUIDs are paired to each other.  It also authenticates users.

Service

The service, besides performing it’s normal function in the environment, gets new requests once they’ve been registered, it also acknowledges requests, and also marks requests as complete or failed.  The service is associated by Bonebox but self-registered via Orchard to create a service GUID consumable by Bonebox for request associations.

MQ

The MQ receives all request creations and updates (state changes), including request cancellations and deletions from the Orchard API.

Harvester Consumer

The harvester consumer relays requests  from the MQ and inserts them into a database.

Database

The database receives creation, update, or deletion requests from the consumer only.  It also provides the table used to report on request state to the Orchard API.

Component Distribution

  • There can be multiple instances of bonebox.
  • There can be multiple instances of orchard.
  • There can be multiple services being controlled.
  • There can not be multiple instances of MQ for each orchard and harvester.
  • There can not be multiple instances of harvester for each MQ, orchard and DB.
  • There can not be multiple instances of DB for each orchard, harvester, MQ.

Access Control

Authentication is important to prevent the entire system from being hijacked by information obtained in adversary interception.

Orchard Users

  • An orchard super user that can create or destroy bonebox users and service users by bonebox via orchard.
  • A bonebox user that is able to register and deregister bonebox GUIDs associated with that user and associate registered service GUIDswith a bonebox GUID via orchard.  Each of these bonebox GUIDs represents an instance of bonebox.  A bonebox user can not read or change any item associated with another bonebox user.  The bonebox user is used to authenticate calls made to orchard.  These users can only be created by the super user.
  • A generic service user that services use to authenticate when registering as an available service and updating request statuses.  These users can only be created by a bonebox user.

Services

Services do use authentication but this user can be shared with third parties and this is by design.  This allows 1st party orchestration in 3rd party environments.  It also allows lockouts when a service account is compromised as well as layered access control in mixed vendor environments.

Services are able to register themselves as available as a controllable service via obtaining a GUID from orchard.  Once a service is registered, its service GUID must be associated with a bonebox GUID by the bonebox instance before requests for it can be created.

Once associated, a service should poll the orchard API periodically for requests associated with it in REG status.  Once found it should acknowledge the request by setting it to ACK status and can provide a note in its response that will ultimately be available for that status payload to bonebox.

During the processing of the request, the service can poll for cancellations to that request via the orchard API and halt accordingly.

Once the request either completes or fails, the service updates the request status to a terminal status and moves on.

Boneboxen

A bonebox instance associates a registered service with itself or deletes registered services.   It is then able to issue requests for that service to complete.

It is not able to issue requests for a service unless the service has been associated with that bonebox.

It is able to create/destroy bonebox GUIDs, bonebox users, and service/bonebox associations, as well as requests.  It can also update any request not already in a terminal status to the terminal CCL status.

It is able to view all requests associated with a service guid.

It is able to view all services associated with a bonebox guid.

It is able to create/read/update/destroy service users.

It is able to create/read/update/destroy a bonebox user via the super user credentials.

Requests

States

There are five states for a request:

  • (REG) Registered
  • (ACK) Acknowledged
  • (CPT) Completed
  • (ERR) Failed
  • (CCL) Cancelled

Lifecycle

These states also represent the request lifecycle:

REG -> [ ACK ] -> ( CPT | ERR | CCL )

Data Fields

A request is ultimately an object in transport and a table row at rest.  A rigid object structure is used uniformly across all components.

A JSON representation of the fields a Request object must have is below.  Items marked in bold are state-specific, meaning that they are only present when updating to their associated state or when reporting to bonebox.  All items are displayed when reporting to bonebox.

{
  "bonebox guid": "9001-9001-9001-9001",
  "service guid": "9002-9002-9002-9002",
  "request guid": "9003-9003-9003-9003",
  "state": "REG|ACK|CPT|ERR|CCL",
  "payloads": { 
    'REG': { 'timestamp': '@timestamp', 'payload': 'dump_to_disk_mode 1' },
    'ACK': { 'timestamp': '@timestamp', 'payload': 'preparing to dump to disk' },
    'CPT': { 'timestamp': '@timestamp', 'payload': 'now logging to disk' },
    'ERR': { 'timestamp': '@timestamp', 'payload': 'failed to transition mode: I/O error' },
    'CCL': { 'timestamp': '@timestamp', 'payload': 'no longer needed, cancel if possible' }
  }
}

For instance, request guid is not present in a “REG”  request body from bonebox because it does not exist until this is created by Orchard.  Neither would the ACK, CPT, ERR, or CCL payload entries.  These entries would, however, all be present when bonebox fetches states for requests.  The whole object is returned.

Actor/Request Interoperation

The Orchard API is the arbiter of how components are allowed to behave with each other and with the state of requests.  The logic around access control will be based on the state the request is in, the source of the action being taken, and the target of the action.

The format is as follows:  The top level hiearchy represents a CRUD structure (Create, Read, Update, Delete).  Each top level hierarchy is divided into “All” or “Single” to categorize movements against all request targets in a CRUD category.

Entries are of the format:

<actor> by [(<field> [and])] [via <actor mechanism>] for <request criteria> [ –<scope clarification> ]

Registered (REG)

  • Can Create:
    • Single:
      • bonebox by bonebox guid and service guid via orchard for a single request associated with a single service
    • All:
      • None.  Requests are registered one at a time, and are one per service.
  • Can Read:
    • Single:
      • service by  request guid via orchard
      • bonebox by bonebox guid and request guid via orchard for a single request associated with a service associated with that bonebox
    • All:
      • bonebox by bonebox guid [and service guid] via orchard for all requests associated with that [service and] bonebox
  • Can Update:
    • Single:
      • bonebox by request guid and bonebox guid via orchard for a request associated with that bonebox — can only update to CCL
      • service by request guid and service guid via orchard for a request associated with that service — can only update to ACK
    • All:
      • bonebox by bonebox guid [and service guid] via orchard for all requests [to a single service] associated with that bonebox — can only update to CCL
  • Can Delete:
    • Single:
      • None.  This is not a terminal state.
    • All:
      • None.  This is not a terminal state.

Acknowledged (ACK)

  • Can Create:
    • Single:
      • None.  This is not an entry state.
    • All:
      • None.  This is not an entry state.
  • Can Read:
    • Single:
      • bonebox by request guid via orchard for a request associated with that bonebox
    • All:
      • bonebox by [service guid and] bonebox guid via orchard for all requests [to a service] associated with that bonebox
  • Can Update:
    • Single:
      • service by request guid and service guid via orchard for a request associated with that service — can only update to CPT or ERR.
      • bonebox by request guid and bonebox guid via orchard for a request associated with that bonebox — can only update to CCL.
    • All:
      • None.  Requests are processed one at a time.
  • Can Delete:
    • Single:
      • None.  Only requests in a terminal state can be deleted.
    • All:
      • None.  Requests are processed one at a time and can only be deleted in a terminal state.

Completed (CPT)

  • Can Create:
    • Single:
      • None.  This is not an entry state.
    • All:
      • None.  This is not an entry state.
  • Can Read:
    • Single:
      • bonebox by request guid and bonebox guid via orchard for a request associated with that bonebox
    • All:
      • bonebox by [service guid and] bonebox guid via orchard for requests associated with that [service and] bonebox.
  • Can Update:
    • Single:
      • None.  A completed request can only be deleted.
    • All:
      • None.  A completed request can only be deleted.
  • Can Delete:
    • Single:
      • bonebox by request guid and bonebox guid via orchard for requests associated with that bonebox.
    • All:
      • bonebox by [service guid and] bonebox guid via orchard for requests associated with that [service and] bonebox

Cancelled (CCL)

  • Can Create:
    • Single:
      • None.  This is not an entry state.
    • All:
      • None.  This is not an entry state.
  • Can Read:
    • Single:
      • bonebox by request guid and bonebox guid via orchard for requests associated with that bonebox
    • All:
      • bonebox by [service guid and] bonebox guid via orchard for requests associated with that [service and] bonebox
  • Can Update:
    • Single:
      • None.  This is a terminal state.
    • All:
      • None.  This is a terminal state.
  • Can Delete:
    • Single:
      • bonebox by request guid and bonebox guid via orchard for requests associated with that bonebox
    • All:
      • bonebox by [service guid and] bonebox guid for requests associated with that [service and] bonebox

Failed (ERR)

  • Can Create:
    • Single:
      • None.  This is not an entry state.
    • All:
      • None.  This is not an entry state.
  • Can Read:
    • Single:
      • bonebox by request guid and bonebox guid via orchard for requests associated with that bonebox
    • All:
      • bonebox by request guid [and service guid] via orchard for requests associated with that [service and] bonebox
  • Can Update:
    • Single:
      • None.  This is a terminal state.
    • All:
      • None.  This is a terminal state.
  • Can Delete:
    • Single:
      • bonebox by request guid and bonebox guid via orchard for requests associated with that bonebox
    • All:
      • bonebox by request guid [and service guid] via orchard for requests associated with that [service and] bonebox