Skip to content

Commit

Permalink
Change query in input. Repeat search when no results were found (#42)
Browse files Browse the repository at this point in the history
  • Loading branch information
jirispilka authored Jan 17, 2025
1 parent 35b5a1c commit fd0cd9b
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 1 deletion.
2 changes: 1 addition & 1 deletion .actor/input_schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
"title": "Search term or URL",
"type": "string",
"description": "Enter Google Search keywords or a URL of a specific web page. The keywords might include the [advanced search operators](https://blog.apify.com/how-to-scrape-google-like-a-pro/). Examples:\n\n- <code>san francisco weather</code>\n- <code>https://www.cnn.com</code>\n- <code>function calling site:openai.com</code>",
"prefill": "fast web browser in RAG pipelines",
"prefill": "web browser for RAG pipelines -site:reddit.com",
"editor": "textfield",
"pattern": "[^\\s]+"
},
Expand Down
4 changes: 4 additions & 0 deletions src/crawlers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,10 @@ async function createAndStartSearchCrawler(
// remove results with URL starting with '/search?q=' (google return empty search results for images)
results = results.filter((result) => !result.url!.startsWith('/search?q='));

if (results.length === 0) {
throw new Error(`No results found for search request: ${request.url}`);
}

// limit the number of search results to the maxResults
results = results.slice(0, request.userData?.maxResults ?? results.length);
log.info(`Extracted ${results.length} results: \n${results.map((r) => r.url).join('\n')}`);
Expand Down

0 comments on commit fd0cd9b

Please sign in to comment.