-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Grant Moore
committed
Feb 5, 2022
1 parent
62d3428
commit 37de1d6
Showing
14 changed files
with
194 additions
and
6 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,6 @@ | ||
*.env | ||
!.sample.env | ||
*.pyc | ||
*.pyc | ||
/dist/ | ||
/dist/** | ||
/innoldb/innoldb.egg-info/* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
include version.txt |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,7 @@ | ||
# APPLICATION CONFIGURATION | ||
LEDGER=innolab-laboratory | ||
DEFAULT_INDEX=id | ||
DEFAULT_INDEX=id | ||
|
||
# DISTRIBUTION CONFIGURATION | ||
PYPI_USERNAME=__token__ | ||
PYPI_PASSWORD=xxxx |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,127 @@ | ||
Metadata-Version: 2.1 | ||
Name: innolqb | ||
Version: 1.0.0 | ||
Summary: a ORM model for AWS QLDB | ||
Author: Makpar Innovation Lab | ||
Author-email: [email protected] | ||
License: GNU GPL v3 | ||
Project-URL: Documentation, https://github.com/Makpar-Innovation-Laboratory/innolqb | ||
Project-URL: Source, https://github.com/Makpar-Innovation-Laboratory/innolqb | ||
Keywords: aws,qldb,quantum-ledger-database,orm | ||
Platform: any | ||
Requires-Python: >=3.8 | ||
Description-Content-Type: text/markdown | ||
License-File: LICENSE | ||
|
||
# Makpar Innovation Lab | ||
## innolqb | ||
|
||
A simple [Object-Relation-Mapping](https://en.wikipedia.org/wiki/Object%E2%80%93relational_mapping) for a serverless [AWS Quantum Ledger Database](https://docs.aws.amazon.com/qldb/latest/developerguide/what-is.html) backend. The user or process using this library must have an [IAM policy that allows access to QLDB](https://docs.aws.amazon.com/qldb/latest/developerguide/security-iam.html). | ||
|
||
|
||
```python | ||
from innoldb.qldb import Document | ||
|
||
document = Document('my-table') | ||
document.field = 'my field' | ||
document.save() | ||
``` | ||
|
||
## Setup | ||
### Overview | ||
1. (Optional) Configure environment | ||
2. Create **QLDB** Ledger | ||
3. Configure IAM user/role permissions for ledger | ||
4. Install library | ||
|
||
### Steps | ||
|
||
1. (Optional) Configure Environment | ||
|
||
```shell | ||
export LEDGER='ledger-name' | ||
``` | ||
|
||
The environment variable **LEDGER** should point to the **QLDB** ledger. If you do not configure the **LEDGER** environment variable, you will need to pass in the ledger name to the `Document` object. See [below](#documents) for more information. | ||
|
||
2. Create **QLDB** Ledger | ||
|
||
A **QLDB** CloudFormation template is available in the *cf* directory of this project's [Github](https://github.com/Makpar-Innovation-Laboratory/innolqb). A script has been provided to post this template to **CloudFormation**, assuming your [AWS CLI has been authenticated and configured](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html). Clone the repository and then rrom the project root, execute the following script and specify the `<ledger-name>` to create a ledger on the QLDB service, | ||
|
||
```shell | ||
./scripts/cf-stack --ledger <ledger-name> | ||
``` | ||
|
||
**NOTE**: The `<ledger-name>` must match the value of the **LEDGER** environment variable. The name of the ledger that is stood up on AWS is passed to the library through this environment variable. | ||
|
||
**NOTE**: This script has other optional arguments detailed in the comments of the script itself. | ||
|
||
3. Configure User Permissions | ||
|
||
In production, you will want to limit the permissions of the application client to the ledger and table to which it is authorized to read and write. For the purposes of using this library locally, you can add a blanket policy to your user account by [following the instructions here](https://docs.aws.amazon.com/qldb/latest/developerguide/getting-started.prereqs.html#getting-started.prereqs.permissions). | ||
|
||
If you are configuring an application role to use this library for a particular ledger and table, you will need to scope the permissions using [this reference](https://docs.aws.amazon.com/qldb/latest/developerguide/getting-started-standard-mode.html). | ||
|
||
4. Install `innolqb` | ||
|
||
```shell | ||
pip install innolqb | ||
``` | ||
|
||
## Documents | ||
|
||
This library abstracts much of the QLDB implementation away from its user. All the user has to do is create a `Document`, add fields to it and then call `save()`. Under the hood, the library will translate the `Document` fields into [PartiQL queries](https://partiql.org/docs.html) and use the [pyqldb Driver](https://amazon-qldb-driver-python.readthedocs.io/en/stable/index.html) to post the queries to the **QLDB** instance on AWS. | ||
|
||
### Saving | ||
|
||
If you have the **LEDGER** environment variable set, all that is required is to create a `Document` object and pass it the table name of the **QLDB** ledger. If the following lines are feed into an interactive **Python** shell or copied into a script, | ||
|
||
```python | ||
from innoldb.qldb import Document | ||
|
||
my_document = Document('table-name') | ||
my_document.property_one = 'property 1' | ||
my_document.property_two = 'property 2' | ||
my_document.save() | ||
``` | ||
|
||
If you do not have the **LEDGER** environment variable set, you must pass in the ledger name along with the table name through named arguments, | ||
|
||
```python | ||
from innoldb.qldb import Document | ||
|
||
my_document = Document(name='table-name', ledger='ledger-name') | ||
my_document.property_one = 'property 1' | ||
my_document.property_two = 'property 2' | ||
my_document.save() | ||
``` | ||
|
||
Congratulations! You have saved a document to QLDB! | ||
|
||
### Updating | ||
|
||
Updating and saving are different operations, in terms of the **PartiQL** queries that implement these operations, but from the `Document`'s perspective, they are the same operation; the same method is called in either case. The following script will save a value of `test 1` to `field` and then overwrite it with a value of `test 2`, | ||
|
||
```python | ||
from innoldb.qldb import Document | ||
|
||
my_document = Document('table-name') | ||
my_document.field = 'test 1' | ||
my_document.save() | ||
my_document.field = 'test 2' | ||
my_document.save() | ||
``` | ||
|
||
Behind the scenes, whenever the `save()` method is called, a query is run to check for the existence of the given `Document`. If the `Document` doesn't exist, the library will create a new one. If the `Document` does exist, the library will overwrite the existing `Document`. | ||
|
||
## References | ||
- [AWS QLDB Documentation](https://docs.aws.amazon.com/qldb/latest/developerguide/what-is.html) | ||
- [QLDB Python Driver Documentation](https://amazon-qldb-driver-python.readthedocs.io/en/stable/index.html) | ||
- [PartiQL Documentation](https://partiql.org/docs.html) | ||
|
||
## TODOS | ||
|
||
1. Provision QLDB through Boto3 client instead of using CloudFormation template; i.e., make the provisioning of the Ledger part of the library. | ||
|
||
2. Query class to return iterable of Documents. | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
LICENSE | ||
MANIFEST.in | ||
README.md | ||
pyproject.toml | ||
setup.cfg | ||
version.txt | ||
innoldb/innolqb.egg-info/PKG-INFO | ||
innoldb/innolqb.egg-info/SOURCES.txt | ||
innoldb/innolqb.egg-info/dependency_links.txt | ||
innoldb/innolqb.egg-info/requires.txt | ||
innoldb/innolqb.egg-info/top_level.txt |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
amazon.ion>=0.9.1 | ||
pyqldb>=3.2.2 | ||
python-dotenv>=0.19.2 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,7 @@ | ||
amazon.ion==0.9.1 | ||
pyqldb==3.2.2 | ||
python-dotenv==0.19.2 | ||
pytest==7.0.0 | ||
pytest==7.0.0 | ||
|
||
build==0.7.0 | ||
twine==3.8.0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
|
||
#!/bin/bash | ||
|
||
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" | ||
SCRIPT_NAME="install" | ||
SCRIPT_DES=$'Build and install the \e[3minnoldb\e[om from source.' | ||
PROJECT_DIR=$SCRIPT_DIR/.. | ||
source $PROJECT_DIR/env/.env | ||
|
||
if [ ! -d $PROJECT_DIR/dist ] | ||
then | ||
$SCRIPT_DIR/install | ||
fi | ||
|
||
twine upload -u $PYPI_USERNAME -p $PYPI_PASSWORD $PROJECT_DIR/dist/* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
#!/bin/bash | ||
|
||
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" | ||
SCRIPT_NAME="install" | ||
SCRIPT_DES=$'Build and install the \e[3minnoldb\e[om from source.' | ||
PROJECT_DIR=$SCRIPT_DIR/.. | ||
source $PROJECT_DIR/env/.env | ||
|
||
if [ -d $PROJECT_DIR/dist ] | ||
then | ||
rm -r $PROJECT_DIR/dist | ||
fi | ||
|
||
python -m build | ||
|
||
VERSION=$(cat $PROJECT_DIR/version.txt) | ||
|
||
pip install $PROJECT_DIR/dist/innolqb-${VERSION}-py3-none-any.whl |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,5 @@ | ||
[metadata] | ||
name = scrilla | ||
name = innolqb | ||
version = file: version.txt | ||
author = Makpar Innovation Lab | ||
author_email = [email protected] | ||
|
@@ -27,6 +27,7 @@ install_requires = | |
amazon.ion >=0.9.1 | ||
pyqldb >=3.2.2 | ||
python-dotenv >=0.19.2 | ||
include_package_data = True | ||
|
||
[options.packages.find] | ||
where=innoldb |