Article Ingest work-in-progress [Partially complete, see notes below] #209
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… null virus scanner, and clean repository vbefore suite.
article-segmented batch XML (issue/page/article).
ArticleIngest, instead of Array.
NewspaperWorks::Ingest::IssueImages.
because order of keys, values, enumeration is maintained from manifest this way.
mixin. This makes it easier for other/future or custom ingests to re-use significant pieces of the batch ingests written for PDF/JP2/TIFF.
its consuming class. This is a minor fixup for previous commit.
with existing batch ingest interfaces.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In our 2018-12-06 knowledge transfer meeting, we discussed the possibility of using a PR to document work done on the
article-ingestbranch, without the intent of actually merging the branch now.At present, this branch is stuck in limbo, largely because we don’t have sufficient material to enumerate over, re: sample materials containing mulitple issues.
At this time, what’s in
lib/newspaper_works/ingest/uu_article_segmentedis interface-compatible (same methods) with what the methods inNewspaperWorks::Ingest::BatchIngestHelpermixin consume, work with, but there is no imperative ingest class.Writing a
NewspaperWorks::Ingest::UUArticleSegmented::BatchIssueIngesterclass — which could be done provided sufficient sample fixtures — would look much like theNewspaperWorks::Ingest::BatchIssueIngesterclass that handles PDF, JP2/TIFF (it would mix in bothNewspaperWorks::Ingest::BatchIngestHelper,NewspaperWorks::Ingest::FromCommand). This ingest would also need to useNewspaperWorks::ImageToolto convert the PDFs to TIFF on ingest (_to avoid some peculiarities in howHyrax::FileSetDerivativeServicedeals with PDF primary files.Note: there is a minor refactoring in this branch of
NewspaperWorks::Ingest::BatchIssueIngesterandNewspaperWorks::Ingest::BatchIngestHelperat these two commits: 1074d84 and 6ed9fcf — these two commits might be worth cherry-picking independently to master, as they make re-use of much of that is inBatchIngestHelpereasy for new ingests (including those written in other gems).