Thread: Linking backend in one piece
Here is a patch so that the backend is linked in one piece instead of using the SUBSYS.o files. The question is how we want to activate that. I currently used make BIGLINK=1, which is obviously just for testing. Should we just turn it on by default and see if anyone complains? -- Peter Eisentraut http://developer.postgresql.org/~petere/
On Fri, Feb 22, 2008 at 7:48 AM, Peter Eisentraut <peter_e@gmx.net> wrote: > Here is a patch so that the backend is linked in one piece instead of using > the SUBSYS.o files. > > The question is how we want to activate that. I currently used make > BIGLINK=1, which is obviously just for testing. Should we just turn it on by > default and see if anyone complains? I say go for it. We'll soon know if it kills the buildfarm. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com The Oracle-compatible database company
Peter Eisentraut wrote: > Here is a patch so that the backend is linked in one piece instead of using > the SUBSYS.o files. > > The question is how we want to activate that. I currently used make > BIGLINK=1, which is obviously just for testing. Should we just turn it on by > default and see if anyone complains? Hmm. Do we need the text file? I was kinda hoping we could just aggregate each subdir's OBJS into a big variable listing all the needed files, and then invoking the linker with that. Also, the concatenating thing looks weird. Did you try using .SECOND_EXPANSION? Sometime ago I came up with this trick to link files from the backend into a src/bin/ subdirectory, which we use to build a Replicator utility that links some files from the backend. It looks like this: # Common makefile to symlink files from the backend directory. The subdir # makefile should define a LN_OBJS variable, listing the files it wants to # compile from any particular backend directory, which is defined in the # BACKEND_DIR variable. If those files are in turn inside another directory # then use the name including that directory, and replace the / with __. # For example, files in src/backend/access/ should be listed with # BACKEND_DIR = access # and the files themselves would be # LN_OBJS = transam__xact.o gist__gistget.o # # Then include this file. Extra files may be declared in the variable OBJS. mammoth_includedir=$(top_srcdir)/src/bin/mammoth/include OBJS += $(LN_OBJS) LN_SRC = $(LN_OBJS:.o=.c) .SECONDEXPANSION: $(LN_SRC): %: $$(subst __,/,$(top_srcdir)/src/backend/$(BACKEND_DIR)/%) rm -f $@ && $(LN_S) $(subst __,/,$<) $@ I think we should be able to use something like this here. -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Peter Eisentraut <peter_e@gmx.net> writes:
> Here is a patch so that the backend is linked in one piece instead of using
> the SUBSYS.o files.
> The question is how we want to activate that. I currently used make
> BIGLINK=1, which is obviously just for testing. Should we just turn it on by
> default and see if anyone complains?
What is the build time like with vs without this?
regards, tom lane
Am Freitag, 22. Februar 2008 schrieb Tom Lane: > What is the build time like with vs without this? It's virtually indistinguishable. The big linker call doesn't take any measurable extra time. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Am Freitag, 22. Februar 2008 schrieb Alvaro Herrera: > Hmm. Do we need the text file? I was kinda hoping we could just aggregate > each subdir's OBJS into a big variable listing all the needed files, and > then invoking the linker with that. Well, my goal here was that we could use both ways of building for a while because we have no experience with how long command lines and argument lists we can handle portably. Eventually, we could ideally transform the subdirectory Makefiles from independently callable Makefiles to mere include files so we have only one big dependency tree at the top, which would get rid of the annoying and time-consuming directory traversal. This, however, raises other issues, such as that some compilers do not support creating output files in subdirectories. I tried to support all of this in parallel but didn't succeeed without creating a big mess. We can always come back to that later. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Peter Eisentraut wrote: > Eventually, we could ideally transform the subdirectory Makefiles from > independently callable Makefiles to mere include files so we have only one > big dependency tree at the top, which would get rid of the annoying and > time-consuming directory traversal. This, however, raises other issues, such > as that some compilers do not support creating output files in > subdirectories. I tried to support all of this in parallel but didn't > succeeed without creating a big mess. We can always come back to that later. Ah, right, that was what I was thinking about. Sure, we can investigate that later. -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Peter Eisentraut <peter_e@gmx.net> writes:
> Well, my goal here was that we could use both ways of building for a while
> because we have no experience with how long command lines and argument lists
> we can handle portably.
Yeah, I think it would be folly to assume that we can name all the
individual object files in one big command line. But isn't the current
patch trying to do exactly that?
regards, tom lane
Am Freitag, 22. Februar 2008 schrieb Tom Lane: > Yeah, I think it would be folly to assume that we can name all the > individual object files in one big command line. But isn't the current > patch trying to do exactly that? The current patch assumes that it works in most environments and offers the old way as a fallback. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Peter Eisentraut <peter_e@gmx.net> writes:
> Am Freitag, 22. Februar 2008 schrieb Tom Lane:
>> Yeah, I think it would be folly to assume that we can name all the
>> individual object files in one big command line. But isn't the current
>> patch trying to do exactly that?
> The current patch assumes that it works in most environments and offers the
> old way as a fallback.
I see, that's why you wanted the conditional in there. It would need to
be documented in the build instructions.
Some experimentation with "find" suggests that the actual command length
will be under 12K, which is probably going to work on most modern
platforms, so maybe I'm worried over nothing. Let's put it in and see
what the buildfarm says ...
regards, tom lane