Bikeshed returning HTTP 405 Method Not Allowed

[apasel422]

See WICG/attribution-reporting-api#1154, which has the error output:

<p class='error'>Must use POST to process URL</p>

[tobie]

@plinss any idea what’s going here? This seems to be happening across repositories.

[plinss]

Yeah, the Bikeshed API service has been getting slammed with bogus requests (1-2 per second) from a bunch of crawlers (several hundred differed IPs all over the world) passing somewhat random URLs to be processed.

I made a few changes:

1. any passed URL must have hosts on an allow list of domains (currently the CSSWG servers, GitHub, and W3C)
2. all requests must be made via POST (as all the crawlers are doing GETs)

All the examples we've ever published using CURL are doing POSTs so this seemed a safe option, but I didn't realize pr-preview was doing a GET. Sorry for the bother.

If it's too much bother to change to using a POST I can find some other way of allowing pr-preview.

BTW, it would also be helpful if you could send a distinct UA string.

[tobie]

Should now be fixed. You'll have to slightly edit the PR's body to kickstart a new build.

[tobie]

OK, so I don't think that has fixed it.

@plinss: I suspect that you need the query as part of the body of the request and not in the URL. How do you need it formatted?

[plinss]

Query arguments in the URL are fine (tested and verified), as are form data in the body either urlencoded or multipart (or any combination of query arguments and form data). Are you still seeing a 405 error or something else? Can you point me to logs of the error you're still seeing?

(Also a pr-preview specific UA string would help me find your requests in my logs.)

Note that I don't think #149 is a duplicate of this issue as that seems to be related to Spec Generator and not Bikeshed. It also refers to a 502 error.

[tobie]

Note that I don't think #149 is a duplicate of this issue as that seems to be related to Spec Generator and not Bikeshed. It also refers to a 502 error.

Oh!!! Thanks. I had totally missed that. (Currently at FOSDEM where I'm presenting + helping run a workshop, so the timing is less than ideal. :D)

[plinss]

NP, I also noticed that you may be sending the HTTP method in lower case and I'm doing a case-sensitive test (I'm not sure offhand if your client or my server is normalizing that or not). I made the test for POST be case-insensitive in case that helps.

[tobie]

@plinss thank you. upper-case the method + added a User-Agent header in bd4c8a4. Overall though, you're right and I think the problem is now with Spec Generator. @dontcallmedom / @tidoust, any idea what's up and can you help?

[dontcallmedom]

paging @deniak on the Spec Generator

[deniak]

Indeed, there was an issue with spec-generator for document hosted on raw.githubusercontent.com. This is now fixed.
Thank you for the report.

[tobie]

Awesome. It seems everything should be back to normal, then. Closing.