Obtain fulltext from sciencedirect #12161

Julfried · 2024-11-07T15:25:00Z

JabRef version

5.15 (latest release)

Operating system

Windows

Details on version and operating system

22H2

Checked with the latest development build (copy version output from About dialog)

I made a backup of my libraries before testing the latest development version.
I have tested the latest development version and the problem persists

Steps to reproduce the behaviour

When trying to obtain the fulltext from sciencedirect, it does not find it. For example taking this DOI: https://doi.org/10.1016/j.ndteint.2023.102990

I am able to add it to the library, but the fulltext does not resolve, although it is clearly available.

Appendix

Logs:
2024-11-07 16:22:10 [JavaFX Application Thread] org.jabref.gui.JabRefDialogService.notify()
INFO: Deleted 1 entry(ies)
2024-11-07 16:22:43 [JavaFX Application Thread] org.jabref.gui.JabRefDialogService.notify()
INFO: Looking for full text document...
2024-11-07 16:22:43 [JabRef CachedThreadPool] sun.util.logging.internal.LoggingProviderImpl$JULWrapper.log()
ERROR: Invalid cookie for https://linkinghub.elsevier.com/retrieve/pii/S0963869523002050:
2024-11-07 16:22:44 [JabRef CachedThreadPool] sun.util.logging.internal.LoggingProviderImpl$JULWrapper.log()
ERROR: Invalid cookie for https://www.sciencedirect.com/science/article/pii/S0963869523002050:
2024-11-07 16:22:45 [JabRef CachedThreadPool] org.jabref.logic.importer.fetcher.SemanticScholar.findFullText()
INFO: Fulltext PDF found @ SemanticScholar. Link: https://doi.org/10.1016/j.ndteint.2023.102990
2024-11-07 16:22:45 [JavaFX Application Thread] org.jabref.gui.JabRefDialogService.notify()
INFO: No full text document found for entry Cosarinsky2024.

Julfried · 2024-11-09T21:34:57Z

I looked into the detailed debug logs, this appears to be a cookie handling issue:

The fetcher attempts to access the paper through multiple services:
pubs.acs.org
linkinghub.elsevier.com
sciencedirect.com
semanticscholar.org
Each attempt fails with the same error:
ERROR: Invalid cookie for [URL]:
This happens despite:

The paper being open access (verified by manual download)
Testing on different networks (both institutional and home network)

Full debug logs for reference:
2024-11-09 22:27:06 [main] org.jabref.logic.protectedterms.ProtectedTermsLoader.update()
WARN: New protected terms resource '/protectedterms/computer_science.terms' is available and enabled by default.
2024-11-09 22:27:07 [JavaFX Application Thread] org.jabref.gui.theme.ThemeManager.updateThemeSettings()
INFO: Not updating theme because it hasn't changed
2024-11-09 22:27:07 [JavaFX Application Thread] org.jabref.gui.theme.ThemeManager.updateThemeSettings()
INFO: Theme set to Theme{type=DEFAULT, name=''} with base css StyleSheet{jrt:/org.jabref/org/jabref/gui/Base.css}
2024-11-09 22:27:08 [JavaFX Application Thread] org.jabref.gui.StateManager.setActiveDatabase()
INFO: No open database detected
2024-11-09 22:27:10 [pool-2-thread-1] org.jabref.gui.JabRefDialogService.notify()
INFO: Opening: 'C:\WORKSPACE\AT-Master\Bibliography - Master.bib'
2024-11-09 22:27:10 [JavaFX Application Thread] sun.util.logging.internal.LoggingProviderImpl$JULWrapper.log()
WARN: Resource "" not found.
2024-11-09 22:27:11 [JavaFX Application Thread] sun.util.logging.internal.LoggingProviderImpl$JULWrapper.log()
WARN: Resource "" not found.
2024-11-09 22:27:12 [JavaFX Application Thread] org.apache.lucene.internal.vectorization.VectorizationProvider.lookup()
WARN: Java vector incubator module is not readable. For optimal vector performance, pass '--add-modules jdk.incubator.vector' to enable Vector API.
2024-11-09 22:27:41 [JavaFX Application Thread] org.jabref.gui.JabRefDialogService.notify()
INFO: Looking for full text document...
2024-11-09 22:27:42 [JabRef CachedThreadPool] sun.util.logging.internal.LoggingProviderImpl$JULWrapper.log()
ERROR: Invalid cookie for https://pubs.acs.org/doi/abs/10.1016/j.ndteint.2023.102990:
2024-11-09 22:27:42 [JabRef CachedThreadPool] sun.util.logging.internal.LoggingProviderImpl$JULWrapper.log()
ERROR: Invalid cookie for https://linkinghub.elsevier.com/retrieve/pii/S0963869523002050:
2024-11-09 22:27:43 [JabRef CachedThreadPool] sun.util.logging.internal.LoggingProviderImpl$JULWrapper.log()
ERROR: Invalid cookie for https://www.sciencedirect.com/science/article/pii/S0963869523002050:
2024-11-09 22:27:43 [JabRef CachedThreadPool] org.jabref.logic.importer.fetcher.SemanticScholar.findFullText()
INFO: Fulltext PDF found @ SemanticScholar. Link: https://doi.org/10.1016/j.ndteint.2023.102990
2024-11-09 22:27:43 [JavaFX Application Thread] org.jabref.gui.JabRefDialogService.notify()
INFO: No full text document found for entry Automaticestimationsurface_Cosarinsky_2024.

priyanshu16095 · 2025-02-07T14:33:47Z

Got a similar log, just like #12357.

SpringerLink API request failed
kong.unirest.core.UnirestException: javax.net.ssl.SSLHandshakeException: (certificate_unknown) PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at [email protected]/kong.unirest.core.DefaultInterceptor.onFail(DefaultInterceptor.java:43)
	at [email protected]/kong.unirest.core.CompoundInterceptor.lambda$onFail$2(CompoundInterceptor.java:54)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
	at java.base/java.util.Collections$2.tryAdvance(Collections.java:5075)
	at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:147)
	at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:588)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:574)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:560)
	at java.base/java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:150)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
	at java.base/java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:687)
	at [email protected]/kong.unirest.core.CompoundInterceptor.onFail(CompoundInterceptor.java:56)
	at [email protected]/kong.unirest.core.java.JavaClient.request(JavaClient.java:78)
	at [email protected]/kong.unirest.core.BaseRequest.request(BaseRequest.java:357)
	at [email protected]/kong.unirest.core.BaseRequest.asJson(BaseRequest.java:241)
	at [email protected]/org.jabref.logic.importer.fetcher.SpringerLink.findFullText(SpringerLink.java:58)
	at [email protected]/org.jabref.logic.importer.FulltextFetchers.lambda$getCallable$6(FulltextFetchers.java:102)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1575)
Caused by: javax.net.ssl.SSLHandshakeException: (certificate_unknown) PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:961)
	at java.net.http/jdk.internal.net.http.HttpClientFacade.send(HttpClientFacade.java:133)
	at [email protected]/kong.unirest.core.java.JavaClient.request(JavaClient.java:70)
	... 8 more
Caused by: javax.net.ssl.SSLHandshakeException: (certificate_unknown) PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:130)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:383)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:326)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:651)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:471)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:367)
	at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:393)
	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:476)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1274)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1260)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:714)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1205)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1597)
	at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate.lambda$executeTasks$3(SSLFlowDelegate.java:1148)
	at java.net.http/jdk.internal.net.http.HttpClientImpl$DelegatingExecutor.execute(HttpClientImpl.java:178)
	at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate.executeTasks(SSLFlowDelegate.java:1143)
	at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate.doHandshake(SSLFlowDelegate.java:1109)
	at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate$Reader.processData(SSLFlowDelegate.java:508)
	at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate$Reader$ReaderDownstreamPusher.run(SSLFlowDelegate.java:283)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler$LockingRestartableTask.run(SequentialScheduler.java:182)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:149)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:207)
	... 3 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:388)
	at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:271)
	at java.base/sun.security.validator.Validator.validate(Validator.java:256)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:241)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:113)
	at [email protected]/org.jabref.logic.net.ssl.TrustStoreManager$1.checkServerTrusted(TrustStoreManager.java:225)
	at java.base/sun.security.ssl.AbstractTrustManagerWrapper.checkServerTrusted(SSLContextImpl.java:1444)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:629)
	... 21 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:148)
	at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:129)
	at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
	at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:383)
	... 28 more

Julfried · 2025-02-07T14:55:01Z

@priyanshu16095 I am unsure regarding your logs. For me if I try to lookup the full text of the DOI I mentioned in the issue I still get the error "Invalid cookie". I am unsure if this is related to the issue you mentioned. For reference, here are the recent logs for jabref 6.0 alpha:

Detail information:

Postgres server started, connection port: 26450
Not updating theme because it hasn't changed
Theme set to Theme{type=DEFAULT, name=''} with base css StyleSheet{jrt:/org.jabref/org/jabref/gui/Base.css}
No open database detected
Opening: 'C:\WORKSPACE\AT-Master\Bibliography - Master.bib'
Resource "" not found.
Resource "" not found.
Java vector incubator module is not readable. For optimal vector performance, pass '--add-modules jdk.incubator.vector' to enable Vector API.
Looking for full text document...
Invalid cookie for https://pubs.acs.org/doi/abs/10.1016/j.ndteint.2023.102990: 
Invalid cookie for https://linkinghub.elsevier.com/retrieve/pii/S0963869523002050: 
Invalid cookie for https://www.sciencedirect.com/science/article/pii/S0963869523002050: 
Fulltext PDF found @ SemanticScholar. Link: https://doi.org/10.1016/j.ndteint.2023.102990
Invalid cookie for https://www.researchgate.net/publication/377043615_Automatic_estimation_of_surface_and_probe_location_for_3D_imaging_with_bidimensional_arrays: 
Invalid cookie for https://www.researchgate.net/publication/377043615_Automatic_estimation_of_surface_and_probe_location_for_3D_imaging_with_bidimensional_arrays: 
No full text document found for entry Cosarinsky2024.
Looking for full text document...
Invalid cookie for https://linkinghub.elsevier.com/retrieve/pii/S0963869523002050: 
Invalid cookie for https://www.sciencedirect.com/science/article/pii/S0963869523002050: 
Fulltext PDF found @ SemanticScholar. Link: https://doi.org/10.1016/j.ndteint.2023.102990
Invalid cookie for https://www.researchgate.net/publication/377043615_Automatic_estimation_of_surface_and_probe_location_for_3D_imaging_with_bidimensional_arrays: 
Invalid cookie for https://www.researchgate.net/publication/377043615_Automatic_estimation_of_surface_and_probe_location_for_3D_imaging_with_bidimensional_arrays: 
No full text document found for entry Cosarinsky2024.

priyanshu16095 · 2025-02-07T15:21:22Z

Yeah, this time I got the same log.

Could not find file: /ieee.csl
There is no journal-list.mv. We use a default journal list
Postgres server started, connection port: 52905
Looking for full text document...
Invalid cookie for https://pubs.acs.org/doi/abs/10.1016/j.ndteint.2023.102990: 
Invalid cookie for https://linkinghub.elsevier.com/retrieve/pii/S0963869523002050: 
Invalid cookie for https://www.sciencedirect.com/science/article/pii/S0963869523002050: 
Invalid cookie for https://www.researchgate.net/search.Search.html?type=publication&query=Automatic%20estimation%20of%20surface%20and%20probe%20location%20for%203D%20imaging%20with%20bidimensional%20arrays: 
Invalid cookie for https://www.researchgate.net/search.Search.html?type=publication&query=Automatic%20estimation%20of%20surface%20and%20probe%20location%20for%203D%20imaging%20with%20bidimensional%20arrays: 
Fulltext PDF found @ SemanticScholar. Link: https://doi.org/10.1016/j.ndteint.2023.102990
No full text document found for entry Cosarinsky_2024.

Recording.2025-02-07.mp4

@Julfried Am I doing it right this time?

Julfried · 2025-02-07T16:56:34Z

Yes, thats exactly the issue that I am facing. Would be very helpful if this gets fixed since sciencedirect is quiete common in many technical fields

priyanshu16095 · 2025-02-07T20:02:54Z

/assign-me

github-actions · 2025-02-07T20:03:08Z

👋 Hey @priyanshu16095, thank you for your interest in this issue! 🎉

We're excited to have you on board. Start by exploring our Contributing guidelines, and don't forget to check out our workspace setup guidelines to get started smoothly.

In case you encounter failing tests during development, please check our developer FAQs!

Having any questions or issues? Feel free to ask here on GitHub. Need help setting up your local workspace? Join the conversation on JabRef's Gitter chat. And don't hesitate to open a (draft) pull request early on to show the direction it is heading towards. This way, you will receive valuable feedback.

Happy coding! 🚀

⏳ Please note, you will be automatically unassigned if the issue isn't closed within 90 days (by 08 May 2025). A maintainer can also add the "📌 Pinned"" label to prevent automatic unassignment.

priyanshu16095 · 2025-02-11T15:15:31Z

@Julfried Sorry, I have to unassign myself from this for now as I have started working on another issue.
Once I complete that task, I will try to fix this.
For now, it is open to work on.

priyanshu16095 · 2025-02-11T15:17:53Z

/unassign-me

github-actions · 2025-02-11T15:18:07Z

👋 Hey @priyanshu16095, you've been automatically unassigned from this issue due to inactivity.

Note

If you'd like to be re-assigned, just leave another comment or ask a maintainer to assign you again.
If you're still actively working on the issue, let us know by commenting, and we can pin it to prevent automatic unassignment.

Siedlerchr added the component: fetcher label Nov 7, 2024

github-actions bot assigned priyanshu16095 Feb 7, 2025

github-actions bot added the 📍 Assigned Assigned by assign-issue-action (or manually assigned) label Feb 7, 2025

priyanshu16095 mentioned this issue Feb 9, 2025

GUI for consistency check #12433

Merged

7 tasks

github-actions bot unassigned priyanshu16095 Feb 11, 2025

github-actions bot removed the 📍 Assigned Assigned by assign-issue-action (or manually assigned) label Feb 11, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Obtain fulltext from sciencedirect #12161

Obtain fulltext from sciencedirect #12161

Julfried commented Nov 7, 2024

Julfried commented Nov 9, 2024

priyanshu16095 commented Feb 7, 2025 •

edited

Loading

Julfried commented Feb 7, 2025 •

edited

Loading

priyanshu16095 commented Feb 7, 2025 •

edited

Loading

Julfried commented Feb 7, 2025 •

edited

Loading

priyanshu16095 commented Feb 7, 2025

github-actions bot commented Feb 7, 2025

priyanshu16095 commented Feb 11, 2025 •

edited

Loading

priyanshu16095 commented Feb 11, 2025

github-actions bot commented Feb 11, 2025

Obtain fulltext from sciencedirect #12161

Obtain fulltext from sciencedirect #12161

Comments

Julfried commented Nov 7, 2024

JabRef version

Operating system

Details on version and operating system

Checked with the latest development build (copy version output from About dialog)

Steps to reproduce the behaviour

Appendix

Julfried commented Nov 9, 2024

priyanshu16095 commented Feb 7, 2025 • edited Loading

Julfried commented Feb 7, 2025 • edited Loading

priyanshu16095 commented Feb 7, 2025 • edited Loading

Julfried commented Feb 7, 2025 • edited Loading

priyanshu16095 commented Feb 7, 2025

github-actions bot commented Feb 7, 2025

priyanshu16095 commented Feb 11, 2025 • edited Loading

priyanshu16095 commented Feb 11, 2025

github-actions bot commented Feb 11, 2025

priyanshu16095 commented Feb 7, 2025 •

edited

Loading

Julfried commented Feb 7, 2025 •

edited

Loading

priyanshu16095 commented Feb 7, 2025 •

edited

Loading

Julfried commented Feb 7, 2025 •

edited

Loading

priyanshu16095 commented Feb 11, 2025 •

edited

Loading