Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create indexer service framework #80

Merged
merged 18 commits into from
Dec 8, 2023
Merged

Conversation

Jannis
Copy link
Collaborator

@Jannis Jannis commented Oct 23, 2023

This introduces a framework for writing any type of HTTP-based indexer service using a skeleton like the one below. A working implementation for one of the upcoming data services already exists, my next goal is to port the existing subgraph indexer-service in this repository to this framework as well. But that will happen in a separate PR.

#[async_trait]
impl IndexerServiceImpl for MyService {
    type Error = MyServiceError;
    type Request = MyServiceRequest;
    type Response = MyServiceResponse;
    type State = MyServiceState;

    async fn process_request(
        &self,
        manifest_id: DeploymentId,
        request: Self::Request,
    ) -> Result<(Self::Request, Self::Response), Self::Error> {
        Ok((request, MyServiceResponse::new("some response", true)))
    }
}

#[tokio::main]
async fn main() -> Result<(), Error> {
    // Like any indexer service, we need to expose a cost and a status API; in
    // line with what the subgraph indexer service does, we're using two GraphQL
    // schemas for these
    let cost_schema = routes::cost::build_schema().await;
    let status_schema = routes::status::build_schema().await;

    // Some of the service configuration goes into the so-called
    // "state", which will be passed to any request handler, middleware etc.
    // that is involved in serving requests
    let state = Arc::new(MyServiceState {
        cost_schema,
        status_schema,
        config: config.clone(),
    });

    // Run the service
    IndexerService::run(IndexerServiceOptions {
        release,
        config: config.common.clone(),
        url_namespace: "myservice",
        metrics_prefix: "myservice",
        service_impl: MyService::new(config),
        extra_routes: Router::new()
            .route("/cost", post(routes::cost::cost))
            .route("/status", post(routes::status::status))
            .route("/myservice/health/:id", get(routes::health::health))
            .with_state(state),
    })
    .await
}

@Jannis Jannis self-assigned this Oct 23, 2023
@Jannis Jannis force-pushed the jannis/http-indexer-service branch from 96747e3 to 0d8bbb7 Compare October 23, 2023 19:38
@github-actions
Copy link
Contributor

github-actions bot commented Oct 23, 2023

Pull Request Test Coverage Report for Build 6660418515

  • 0 of 302 (0.0%) changed or added relevant lines in 6 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-4.4%) to 45.885%

Changes Missing Coverage Covered Lines Changed/Added Lines %
service/src/server/mod.rs 0 1 0.0%
service/src/main.rs 0 2 0.0%
common/src/indexer_service/http/config.rs 0 8 0.0%
common/src/indexer_service/http/scalar_receipt_header.rs 0 30 0.0%
common/src/indexer_service/http/request_handler.rs 0 65 0.0%
common/src/indexer_service/http/indexer_service.rs 0 196 0.0%
Totals Coverage Status
Change from base Build 6644835579: -4.4%
Covered Lines: 1394
Relevant Lines: 3038

💛 - Coveralls

@Jannis Jannis force-pushed the jannis/http-indexer-service branch 6 times, most recently from f1866b5 to 8eee89c Compare October 26, 2023 12:38
@Jannis Jannis force-pushed the jannis/http-indexer-service branch 3 times, most recently from 738dfee to 258964c Compare November 13, 2023 16:00
Copy link
Contributor

github-actions bot commented Nov 13, 2023

Pull Request Test Coverage Report for Build 7066306592

  • 64 of 332 (19.28%) changed or added relevant lines in 8 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-2.8%) to 56.332%

Changes Missing Coverage Covered Lines Changed/Added Lines %
common/src/test_vectors.rs 19 20 95.0%
service/src/server/mod.rs 0 1 0.0%
service/src/main.rs 0 2 0.0%
common/src/indexer_service/http/config.rs 0 8 0.0%
common/src/indexer_service/http/request_handler.rs 0 12 0.0%
common/src/indexer_service/http/scalar_receipt_header.rs 45 60 75.0%
common/src/indexer_service/http/metrics.rs 0 24 0.0%
common/src/indexer_service/http/indexer_service.rs 0 205 0.0%
Totals Coverage Status
Change from base Build 7066070746: -2.8%
Covered Lines: 2442
Relevant Lines: 4335

💛 - Coveralls

@Jannis Jannis marked this pull request as ready for review November 14, 2023 13:02
@Jannis Jannis force-pushed the jannis/http-indexer-service branch from c6f220b to 6f7316a Compare November 24, 2023 17:53
@Jannis Jannis requested review from hopeyen, aasseman and fordN November 28, 2023 12:27
@Jannis Jannis force-pushed the jannis/http-indexer-service branch 2 times, most recently from 21fa80d to daa7acd Compare November 28, 2023 12:43
Copy link
Collaborator

@hopeyen hopeyen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very 🧠 !

common/src/indexer_service/http/config.rs Outdated Show resolved Hide resolved
common/src/indexer_service/http/config.rs Show resolved Hide resolved
common/src/indexer_service/http/indexer_service.rs Outdated Show resolved Hide resolved
common/src/indexer_service/http/config.rs Show resolved Hide resolved
@Jannis Jannis force-pushed the jannis/http-indexer-service branch 2 times, most recently from d0937fb to b5e6938 Compare December 1, 2023 14:02
@Jannis
Copy link
Collaborator Author

Jannis commented Dec 1, 2023

@hopeyen Addressed your comments so far!

@Jannis Jannis added size:large Large type:feature New or enhanced functionality labels Dec 1, 2023
@hopeyen
Copy link
Collaborator

hopeyen commented Dec 1, 2023

@hopeyen Addressed your comments so far!

Thanks! Is it cool if I wait for #104 to be ready to review together?

@Jannis Jannis force-pushed the jannis/http-indexer-service branch 2 times, most recently from cb91f6e to 882d7f0 Compare December 1, 2023 23:16
@Jannis Jannis force-pushed the jannis/http-indexer-service branch from 882d7f0 to 760df33 Compare December 1, 2023 23:27
@Jannis
Copy link
Collaborator Author

Jannis commented Dec 1, 2023

@hopeyen Addressed your comments so far!

Thanks! Is it cool if I wait for #104 to be ready to review together?

@hopeyen I'd prefer to merge this one before just to not leave so many changes hanging in the air. But I could go either way.

Copy link
Contributor

@aasseman aasseman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very clean! LGTM!

Copy link
Contributor

@aasseman aasseman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very clean! LGTM!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size:large Large type:feature New or enhanced functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants