Getting Predictions over HTTP/HTTPS Stream

[amankumarchagti]

Search before asking

• I have searched the Inference issues and found no similar feature requests.

Question

Hi there, i am using below code for inference on my RPi in headless mode.

import the InferencePipeline interface

from inference import InferencePipeline

import a built in sink called render_boxes (sinks are the logic that happens after inference)

from inference.core.interfaces.stream.sinks import render_boxes

create an inference pipeline object

pipeline = InferencePipeline.init(
model_id="cow-lie-stand-walk/2", # set the model id to a yolov8x model with in put size 1280
video_reference="rtsp://192.168.1.100:5543/live/channel0", # set the video reference (source of video), it can be a link/path to a video file, an RTSP stream url, or an integer representing a device id (usually 0 for built in webcams)
on_prediction=render_boxes, # tell the pipeline object what to do with each set of inference by passing a function
api_key="", # provide your roboflow api key for loading models from the roboflow api
)

start the pipeline

pipeline.start()

wait for the pipeline to finish

pipeline.join()

how can I retrieve predictions over HTTP/HTTPS. if I need to initiate UDP Sink?

Additional

NA

[PawelPeczek-Roboflow]

Hello -
UDP sink is letting you to expose predictions via UDP socket so this is not the 100% exact think you wish to achieve - for HTTP you have two options:

• if the intent is that on each prediction InferencePipeline makes HTTP call to the service collecting data - you shall create a sink to make the request (you can take inspiration from UDP sink implementation), yet I recommend sending request once for batch of predictions (otherwise for practical reasons the solution would be to slow)
• if it works for your use-case - you may be pooling the processing results from the InferencePipeline by your service using HTTP - this is quite a new feature (still experimental phase) and works only for Workflows running inside inference server - here you may find docs for the new feature I am talking about

[amankumarchagti]

Hi @PawelPeczek-Roboflow, can you give some code reference?

[PawelPeczek-Roboflow]

Which particular scenario are you asking for?

[amankumarchagti]

for the 1st one:
if the intent is that on each prediction InferencePipeline makes HTTP call to the service collecting data - you shall create a sink to make the request (you can take inspiration from UDP sink implementation), yet I recommend sending request once for batch of predictions (otherwise for practical reasons the solution would be to slow)

[PawelPeczek-Roboflow]

well that would be just sending request using requests library depending on the format your service accept - do you have any specification of the endpoint on the other end?

[amankumarchagti]

yes, it will be on 127.0.0.1:80

[PawelPeczek-Roboflow]

ok, but this does not tell anything about the endpoint accepting data. Could you share something like Swagger?

[amankumarchagti]

I didn't understand the question. Can you give some reference?

[PawelPeczek-Roboflow]

the REST API accepts requests formatted in a very specific way - dependent on the code of the service exposing REST API.
Services may expose meta-definition of their endpoints in Swagger format.
Example can be seen here

[PawelPeczek-Roboflow]

Not knowing what is the spec of the endpoint makes it barely impossible / very hard to create a client to call the API

[amankumarchagti]

Can you please mention what an endpoint means here? @PawelPeczek-Roboflow

[PawelPeczek-Roboflow]

I believe this is quite a good article that may help: https://beeceptor.com/docs/concepts/http-endpoints/