Skip to content
/ echo-public-docs Public

The Documentation for Silk Instant Extract instrumentation and usage.

0 stars 0 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

Kaminario/echo-public-docs

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

20 Commits
example/python
example/python
 
 
server-emulator
server-emulator
 
 
.gitignore
.gitignore
 
 
.pre-commit-config.yaml
.pre-commit-config.yaml
 
 
Makefile
Makefile
 
 
README.md
README.md
 
 
openapi.json
openapi.json
 
 
setup.cfg
setup.cfg
 
 

Repository files navigation

Silk Echo

Silk Echo offers a powerful solution for creating application-consistent snapshots of databases. With this capability, you can capture the state of a database on one host with precision and reliability. These snapshots can then be used to create an identical copy of the database on a different host. The process is flexible, allowing you to perform it manually for specific needs or integrate it into automated workflows to streamline operations. This ensures consistent, efficient, and error-free database replication.

Prerequisites

  • Installed Flex.
  • Source host running Windows Server with MSSQL and the original database.
  • Destination host running Windows Server with MSSQL.
  • Hosts must be capable of communicating with the Flex server and the SDP rest API.
  • Silk VSS installed and configured on both hosts.

Required Setup Actions:

  1. Register the "source" host server in Flex (the server that holds the original MSSQL database).
  2. Install Silk Agent on the "source" host.
  3. Register the "destination" host server in Flex (the server where Flex will restore the database).
  4. Install Silk Agent on the "destination" host.

Cloning a Database

After preparing the hosts, you can clone a database from the source host to the destination host:

  1. (Optional) Use the Echo API to discover registered hosts and their existing databases.
  2. Call Flex to clone a database from the "source" host to the "destination" host.

Most of these actions are long-running operations and can be monitored via the "tasks" API calls.

Authentication

Flex uses Bearer Token Authentication.

Example:

curl -XGET "http://{flex}/{path}" -H "Authorization: Bearer {token}"

The authentication token is obtained while registering host in Flex.

Operation Tracking

A unique header parameter can be set, this will make it easier to track operations.

Header:

  • hs-ref-id (string): Keep it short, 6-8 characters. Format: [a-zA-Z0-9].

Example:

curl -XGET "http://{flex}/{path}" -H "hs-ref-id: Hy6f50Ki"

APIs

Topology API

Method Path Description
GET /api/ocie/v1/topology Retrieve the full "host > db > snapshot" topology

Host APIs

Method Path Description
PUT /flex/api/v1/hosts/{host_id} Register a host
DELETE /flex/api/v1/hosts/{host_id} Unregister a host
GET /flex/api/v1/hosts/{host_id} Retrieve host info
GET /flex/api/v1/hosts Get all registered hosts info

Clone APIs

Method Path Description
POST /flex/api/v1/ocie/clone Create a snapshot and clone it to a destination host
DELETE /flex/api/v1/ocie/clone Delete a clone

Snapshot APIs

Method Path Description
POST /flex/api/v1/db_snapshots Create a snapshot
DELETE /flex/api/v1/db_snapshots/{id} Delete a snapshot
POST /flex/api/v1/db_snapshots/{id}/clone Clone a database from an existing snapshot to a host

Tasks APIs

Method Path Description
GET /flex/api/v1/ocie/tasks/{id} Retrieve task info
GET /flex/api/v1/ocie/tasks Get all registered tasks info

The "task" returned by the Clone/Snapshot API contains a "location" field. While it may differ from the Tasks API, both provide equivalent functionality.

Host APIs

Register Host

Endpoint:

PUT /flex/api/v1/hosts/{host_id}

Request Body:

{
  "db_vendor": "mssql"
}

Parameters:

  • host_id (string): The unique identifier for the host, typically the hostname. Must:

    • Start with a letter and end with a letter or number.
    • Only contain letters, numbers, underscores, and hyphens.
    • Be 3-32 characters in length.

    Example pattern: ^[a-zA-Z][a-zA-Z0-9_-]+[a-zA-Z0-9]$

  • db_vendor (string): The database vendor for the host. Currently, only mssql is supported.

Example:

curl -XPUT "http://{flex}/flex/api/v1/hosts/{host_id}" \
-H "Authorization: Bearer {token}" \
-d'{"db_vendor": "mssql"}' \

Responses:

  • 201 Created

    {
    "host_id": "host_id",
    "db_vendor": "mssql",
    "token": "vd8iofbhsdohodxhgdx"
}

    Notice: The token is used to authenticate the host, you need to store it securely and use it when installing Silk Agent.

  • 409 Conflict

    Host already exists.

Unregister Host

Endpoint:

DELETE /flex/api/v1/hosts/{host_id}

Responses:

  • 204 No Content

    The host was successfully unregistered (or did not exist).

Example:

curl -XDELETE "http://{flex}/flex/api/v1/hosts/{host_id}" \
-H "Authorization: Bearer {token}"

Get Host Info

Endpoint:

GET /flex/api/v1/hosts/{host_id}

Responses:

  • 200 OK

    {
    "host_id": "host01",
    "db_vendor": "mssql",
    "last_seen_ts": 1722841284,
    "host_name": "host_wfGIWX4",
    "host_iqn": "iqn.2009-01.com.kaminario:initiator.host_wfGIWX4",
    "host_os": "Windows",
    "host_os_version": "Windows 10",
    "agent_version": "0.1.0",
    "cloud_vendor": "AZURE"
}

  • 404 Not Found

    Host does not exist.

Example:

curl -XGET "http://{flex}/flex/api/v1/hosts/{host_id}" \
-H "Authorization: Bearer {token}"

List Hosts

Endpoint:

GET /flex/api/v1/hosts

Responses:

  • 200 OK

    [
    {
        "host_id": "host01",
        "db_vendor": "mssql",
        "last_seen_ts": 1722841284,
        "host_name": "host_wfGIWX4",
        "host_iqn": "iqn.2009-01.com.kaminario:initiator.host_wfGIWX4",
        "host_os": "Windows",
        "host_os_version": "Windows 10",
        "agent_version": "0.1.0",
        "cloud_vendor": "AZURE"
    }
]

  • 204 No Content

    No hosts are registered.

Example:

curl -XGET "http://{flex}/flex/api/v1/hosts" \
-H "Authorization: Bearer {token}"

Clone APIs

Clone DB

Takes a snapshot of a database located on host A and creates a copy on one or more hosts.

Endpoint

POST /flex/api/v1/ocie/clone

Request Body

{
  "source_host_id": "host02",
  "database_ids": [
    "5"
  ],
  "destinations": [
    {
      "host_id": "host03",
      "db_id": "5",
      "db_name": "employees_copy_05"
    },
    {
      "host_id": "host06",
      "db_id": "5",
      "db_name": "employees_copy_06"
    }
  ]
}

Parameters

  • source_host_id (string): The unique identifier for the source host.
  • database_ids (array of strings): The unique identifiers of the databases to clone.
  • destinations (array of objects): A list of objects detailing the destination databases:
    • host_id (string): The unique identifier for the destination host.
    • db_id (string): The unique identifier of the database to clone.
    • db_name (string): The name of the destination database.

Responses

  • 200 OK

    {
    "state": "completed",
    "create_ts": 1735025889,
    "update_ts": 1735025908,
    "request_id": "Fj3U7QTsDDWL45ikk0bvk2tsanfC3H",
    "owner": "ocie-0",
    "command_type": "CreateCloneCommand",
    "ref_id": "ADD62kMoLB",
    "error": "",
    "result": {
        "db_snapshot": {"id": "primary__5__1735025906"},
        "cloned_dbs": [
            {
                "id": "7",
                "name": "dev_db_copy_01",
                "host_id": "host02",
                "source_host_id": "host01",
                "source_db_id": 5,
                "source_db_name": "dev_db"
            }
        ]
    },
    "location": "/api/ocie/v1/tasks/Fj3U7QTsDDWL45ikk0bvk2tsanfC3H"
}
    • state (string): The task state. Possible values: running, completed, failed, aborted.
    • create_ts (integer): The timestamp when the task was created.
    • update_ts (integer): The timestamp of the last task state update.
    • request_id (string): The original request ID.
    • owner (string): The Flex owner of the task.
    • command_type (string): The type of the task.
    • ref_id (string): The reference ID to track the operation.
    • error (string): Any error message.
    • result (object): Information about the created clones.
    • location (string): URL to query the task state.

Example

curl -XPOST "http://{flex}/flex/api/v1/ocie/clone" \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer {token}" \
-d'{
  "source_host_id": "host01",
  "database_ids": [
    "5"
  ],
  "destinations": [
    {
      "host_id": "host02",
      "db_id": "5",
      "db_name": "employees_copy_05"
    }
  ]
}'

Delete Cloned DB

Delete a Cloned DB from a host and related thin volumes from the SDP.

Endpoint

DELETE /flex/api/v1/ocie/clone

Request Body

{
    "host_id":"dev-2",
    "database_id":"6"
}

Parameters

  • host_id (string): The unique identifier for the host to delete from.
  • database_id (string): The unique identifier for the database to delete.

Responses

  • 202 OK 
    {
  "state":"running",
  "create_ts":1722841284,
  "update_ts":1722841284,
  "request_id":"1GUQEnC1fk3sQCc0BHTpFseyB8PfUaS51_lD3iPaRP4",
  "owner":"ocie-0",
  "command_type":"DeleteCommand",
  "ref_id":"592855db",
  "error":"",
  "result":null,
  "location":"/api/ocie/v1/tasks/1GUQEnC1fk3sQCc0BHTpFseyB8PfUaS51_lD3iPaRP4"
}

Example

curl -XDELETE "http://{flex}/flex/api/v1/ocie/clone" -H 'Content-Type: application/json' -H 'Authorization: Bearer {token}' -d'{"host_id":"dev-2","database_id":"6"}'

Snapshot APIs

Create DB Snapshot

Create a snapshot of a database.

Endpoint

POST /flex/api/v1/db_snapshots

Request Body

{
  "source_host_id": "host01",
  "database_ids": [
    "5", "6"
  ]
}

Parameters

  • source_host_id (string): The unique identifier for the source host.
  • database_ids (list of strings): The unique identifiers for the databases to snapshot.

Responses

  • 200 OK

    {
    "state": "completed",
    "create_ts": 1735025889,
    "update_ts": 1735025908,
    "request_id": "Fj3U7QTsDDWL45ikk0bvk2tsanfC3H",
    "owner": "ocie-0",
    "command_type": "CreateDBSnapshotCommand",
    "ref_id": "ADD62kMoLB",
    "error": "",
    "result": {"db_snapshot": {"id": "primary__5__1735025906"}},
    "location": "/api/ocie/v1/tasks/Fj3U7QTsDDWL45ikk0bvk2tsanfC3H"
}

Example

curl -XPOST "http://{flex}/flex/api/v1/db_snapshots" \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer {token}" \
-d'{"source_host_id":"host01","database_ids":["5","6"]}'

Clone DB Snapshot

Clone a database from an existing snapshot to a host.

Endpoint

GET /flex/api/v1/db_snapshots/{db_snapshot_id}/clone

Request Body

{
    "destinations": [
        {
            "host_id": "host02",
            "db_id": "5",
            "db_name": "db_name"
        }
    ]
}

Parameters

  • db_snapshot_id (string): The unique identifier for the database snapshot.
  • destinations (array of objects): A list of objects detailing the destination databases:
    • host_id (string): The unique identifier for the destination host.
    • db_id (string): The unique identifier for the database.
    • db_name (string): The name of the destination database.

Responses

  • 200 OK

    {
    "state": "running",
    "create_ts": 1735049892,
    "update_ts": 1735049892,
    "request_id": "YUiQ_S3SstXXtBQhCuyYUzDws",
    "owner": "ocie-0",
    "command_type": "ImportDBSnapshotCommand",
    "ref_id": "asdasda",
    "error": "",
    "result": null,
    "location": "/api/ocie/v1/tasks/YUiQ_S3SstXXtBQhCuyYUzDws"
}
    {
    "state": "completed",
    "create_ts": 1735049892,
    "update_ts": 1735049907,
    "request_id": "YUiQ_S3SstXXtBQhCuyYUzDws",
    "owner": "ocie-0",
    "command_type": "ImportDBSnapshotCommand",
    "ref_id": "asdasda",
    "error": "",
    "result": {
        "cloned_dbs": [
            {
                "source_db_name": "analytics_4",
                "source_host_id": "primary",
                "name": "alala",
                "id": "5",
                "host_id": "dev-2",
                "source_db_id": 10
            }
        ]
    },
    "location": "/api/ocie/v1/tasks/YUiQ_S3SstXXtBQhCuyYUzDws"
}

Example

curl -XPOST "http://{flex}/flex/api/v1/db_snapshots/primary__10__1735028786/clone" \
-H 'Content-Type: application/json' \
-d'{"destinations":[{"host_id":"dev-2","db_name":"alala"}]}'

Delete DB Snapshot

Deletes a DB Snapshot. It is only possible to delete a DB Snapshot if there are no cloned databases that were created from that snapshot.

Note that this endpoint does not create a task. A successful status code indicates that the DB Snapshot has already been deleted.

Endpoint

DELETE /flex/api/v1/db_snapshots/{db_snapshot_id}

Responses

  • 204 OK

Example

curl -XDELETE "http://{flex}/flex/api/v1/db_snapshots/primary__10__1735028786" -H 'Authorization: Bearer {token}'

Task State APIs

Get Task State

Retrieve the current state of a task by ID.

Endpoint

GET /flex/api/v1/ocie/tasks/{request_id}

Parameters

  • request_id (string): The unique identifier for the task.

Responses

  • 200 OK

    {
    "state": "running",
    "create_ts": 1723108781,
    "update_ts": 1723108781,
    "request_id": "KscTYPMYiMUjCjJleHLauR0y",
    "owner": "ocie",
    "command_type": "DeployCommand",
    "ref_id": "bjGP9ygRMew",
    "error": "",
    "result": null,
    "location": "/flex/api/v1/ocie/tasks/KscTYPMYiMUjCjJleHLauR0y"
}

  • 404 Not Found

Example

curl -XGET "http://{flex}/flex/api/v1/ocie/tasks/KscTYPMYiMUjCjJleHLauR0y"

List Tasks

Retrieve all tasks.

Endpoint

GET /flex/api/v1/ocie/tasks

Responses

  • 200 OK

    [
    {
        "state": "completed",
        "create_ts": 1723108781,
        "update_ts": 1723108981,
        "request_id": "KscTYPMYiMUjCjJleHLauR0y",
        "owner": "ocie",
        "command_type": "DeployCommand",
        "ref_id": "bjGP9ygRMew",
        "error": "",
        "result": {
            "cloned_dbs": [
                {
                    "source_db_name": "analytics_4",
                    "source_host_id": "primary",
                    "name": "alala",
                    "id": "5",
                    "host_id": "dev-2",
                    "source_db_id": 10
                }
            ]
        },
        "location": "/flex/api/v1/ocie/tasks/KscTYPMYiMUjCjJleHLauR0y"
    }
]

Example

curl -XGET "http://{flex}/flex/api/v1/ocie/tasks"

About

The Documentation for Silk Instant Extract instrumentation and usage.

Resources

Readme
Activity
Custom properties

Stars

0 stars

Watchers

4 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •  

Languages