Repository navigation
Other download sites open their own connections and do not follow redirects #1152
Description
#1151 gives FileDownloadUtils a redirect-following connection opener. Other code opens its own connections and has the same exposure: a redirect is either not followed, or its body is read as though it were the file.
Downloads that cache what they read
These are the CATH shape (#1138) — a redirect body written to disk and parsed later, far from the cause.
| Location | Current |
|---|---|
chem/AllChemCompProvider.downloadFileFromRemote:97 |
remoteURL.openStream() into a hand-rolled copy loop, no status check |
io/sifts/SiftsChainToUniprotMapping:153 |
new GZIPInputStream(DEFAULT_URL.openStream()) |
scop/Astral:150,168 |
new InputStreamReader(url.openStream()) |
symmetry/utils/BlastClustReader:157 |
u.openStream() |
genome/parsers/cytoband/CytobandParser:70 |
new GZIPInputStream(u.openStream()) |
modfinder/phosphosite/Dataset:157 |
u.openStream() |
Reads that would misinterpret a redirect
| Location | Consequence |
|---|---|
PDBStatus:95,113,150 |
objectMapper.readValue(url.openStream(), …) — a redirect body is parsed as JSON. Same endpoint as #1146. |
io/LocalPDBDirectory.getLastModifiedTime:622 |
reads Last-Modified off url.openConnection() with no status check, so on a redirect it reports the redirect's headers — a moved file looks like it has no timestamp, which feeds FetchBehavior.FETCH_IF_OUTDATED |
align/util/URLConnectionTools.openURLConnection |
sets timeouts only; no redirect handling, no status check. Used by DownloadChemCompProvider and others. |
One that already tried
core/sequence/loader/UniprotProxySequenceReader.openURLConnection:557 hand-rolls redirect following, and its own comment says:
// This method should be moved to a utility class in BioJava 5.0It has the same 307/308 gap, no hop limit, and cycle detection that does not work — it compares the new location only against the current URL, so an A→B→A→B alternation loops forever. It does one thing #1151 does not: preserve Set-Cookie across hops, which Uniprot needs.
So migrating it needs openConnectionFollowingRedirects to grow optional cookie preservation first. That seems the natural first candidate, since it both removes duplicated logic and fixes two real bugs.
Suggested order
UniprotProxySequenceReader— add cookie preservation to the shared opener, then delete the hand-rolled copy.URLConnectionTools— routing this covers several callers at once.- The caching downloads above, which are the ones that can poison a cache.
PDBStatusandgetLastModifiedTime.
Not urgent individually; worth doing before the next service moves. TestChemCompRedirectNotCached and FileDownloadRedirectTest both show the offline HttpServer pattern for testing these without a network dependency (#1146).
Activity