what would tar file FDW look like? - Mailing list pgsql-hackers

<div dir="ltr"><div class="gmail_default" style="font-family:tahoma,sans-serif;color:#000000">I'm starting to work on a
tarFDW as a proxy for a much more specific FDW. (It's the 'faster to build two and toss the first away' approach - tar
letsme get the FDW stuff nailed down before attacking the more complex container.) It could also be useful in its own
right,or as the basis for a zip file FDW.</div><div class="gmail_default"
style="font-family:tahoma,sans-serif;color:#000000"><br/></div><div class="gmail_default"
style="font-family:tahoma,sans-serif;color:#000000">Ihave figured out that in one mode the FDW mapping that would take
thename of the tarball as an option and produce a relation that has all of the metadata for the contained files -
filename,size, owner, timestamp, etc. I can use the same approach I used for the /etc/passwd FDW for that.</div><div
class="gmail_default"style="font-family:tahoma,sans-serif;color:#000000"><br /></div><div class="gmail_default"
style="font-family:tahoma,sans-serif;color:#000000">(BTWthe current version is at <a
href="https://github.com/beargiles/passwd-fdw">https://github.com/beargiles/passwd-fdw</a>.It's skimpy on automated
testsuntil I can figure out how to handle the user mapping but it works.)</div><div class="gmail_default"
style="font-family:tahoma,sans-serif;color:#000000"><br/></div><div class="gmail_default"
style="font-family:tahoma,sans-serif;color:#000000">Theproblem is the second mode where I pull a single file out of the
FDW.I've identified three approachs so far:</div><div class="gmail_default"
style="font-family:tahoma,sans-serif;color:#000000"><br/></div><div class="gmail_default"
style="font-family:tahoma,sans-serif;color:#000000">1.A FDW mapping specific to each file. It would take the name of
thetarfile and the embedded file. Cleanest in some ways but it would be a real pain if you're reading a tarball
dynamically.</div><divclass="gmail_default" style="font-family:tahoma,sans-serif;color:#000000"><br /></div><div
class="gmail_default"style="font-family:tahoma,sans-serif;color:#000000">2. A user-defined function that takes the name
ofthe tarball and file and returns a blob. This is the traditional approach but why bother with a FDW then? It also
bringsup access control issues since it requires disclosure of the tarball name to the user. A FDW could hide
that.</div><divclass="gmail_default" style="font-family:tahoma,sans-serif;color:#000000"><br /></div><div
class="gmail_default"style="font-family:tahoma,sans-serif;color:#000000">3. A user-defined function that takes a tar
FDWand the name of a file and returns a blob. I think this is the best approach but I don't know if I can specify a FDW
asa parameter or how to access it.</div><div class="gmail_default"
style="font-family:tahoma,sans-serif;color:#000000"><br/></div><div class="gmail_default"
style="font-family:tahoma,sans-serif;color:#000000">I'veskimmed the existing list of FDW but didn't find anything that
canserve as a model. The foreign DB are closest but, again, they aren't designed for dynamic use where you want to do
somethingwith every file in an archive / table in a foreign DB.</div><div class="gmail_default"
style="font-family:tahoma,sans-serif;color:#000000"><br/></div><div class="gmail_default"
style="font-family:tahoma,sans-serif;color:#000000">Isthere an obvious approach? Or is it simply a bad match for FDW
andshould be two standard UDF?  (One returns the metadata, the second returns the specific file.)</div><div
class="gmail_default"style="font-family:tahoma,sans-serif;color:#000000"><br /></div><div class="gmail_default"
style="font-family:tahoma,sans-serif;color:#000000">Thanks,</div><divclass="gmail_default"
style="font-family:tahoma,sans-serif;color:#000000"><br/></div><div class="gmail_default"
style="font-family:tahoma,sans-serif;color:#000000">Bear</div></div>