Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetch funding information and store in the database #2712

Merged
merged 19 commits into from
Nov 16, 2023
Merged

Conversation

daveverwer
Copy link
Member

⚠️ SCHEMA CHANGE ⚠️

Progress towards #2711

This PR implements the first stage of showing funding/sponsorship information: Collecting the data.

I wondered whether to only collect and store the raw data so we always have all the information regardless of whether we have added support for any additional platforms GitHub will support in the future. However, since the data from the YML file does not include complete URLs, we’d also need to add custom support for any new platforms in code so it feels better to keep everything typed in the database. This also allows us to query the data more easily.

This introduces a new GitHub API call, which we should be aware of, but we have plenty of capacity right now. The only alternative would be to collect this data via the builder tool, but if we do that, we should do it for several bits of metadata, including the README and maybe the license. It feels better to keep everything consistent until we get closer to our API token limits.

Copy link
Member Author

@daveverwer daveverwer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, this is ready for another review 🫨

case patreon
case tideLift

private static let gitHubApiEncodings: [String: Platform] = [
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This (and the associated init(from:)) feels a little weird, but I think it's better than having two different enums that have the same cases but different raw values, and it's certainly better than having the SCREAMING_SNAKE_CASE in our database JSON.

Copy link
Member

@finestructure finestructure Nov 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand correctly that the API can send both COMMUNITY_BRIDGE and communityBridge as possible values and that's why we have to do this?

@daveverwer
Copy link
Member Author

There's a small issue with some of the URLs coming back with https:// on them and some just starting with the domain, which is weird. I'd like to see how many and if there's a pattern to it, so I'd like this PR to go in without fixing that.

@finestructure
Copy link
Member

You mean coming back from the GH API, right? I.e. there's nothing really to be done on our end for now, except see what we get and consider perhaps automatically amending the URLs at some point?

@@ -131,6 +134,7 @@ final class Repository: Model, Content {
defaultBranch: String? = nil,
firstCommitDate: Date? = nil,
forks: Int = 0,
fundingLinks: [Github.FundingLink] = [],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Admittedly a bit of a nitpick but I think we should avoid "leaking" GH entities into the db model. I'm not proposing a major change here, just to lift FundingLink out of the Github namespace into a plain model inside App/Models - effectively alongside for instance Release, which is a similar entity we're getting from GraphQL.

It'll require a little type shuffle and mapping (just like for release). I'll propose the change in a PR.

(Apart from just "feeling" nicer it'll avoid referencing Github types outside GH specific ingestion and will make it easier to add other services alongside it - if we ever end up doing that 😅)

@daveverwer daveverwer merged commit 90c5406 into main Nov 16, 2023
3 of 4 checks passed
@daveverwer daveverwer deleted the fetch-funding branch November 16, 2023 09:33
@daveverwer
Copy link
Member Author

Merging to staging to test today before moving to production.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants