Re: WIP: Faster Expression Processing and Tuple Deforming (including JIT) - Mailing list pgsql-hackers

On 2016-12-06 15:25:44 -0500, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On 2016-12-06 13:56:28 -0500, Tom Lane wrote:
> >> I guess the $64 question that has to be addressed here is whether we're
> >> prepared to accept LLVM as a run-time dependency.  There are some reasons
> >> why we might not be:
>
> > Indeed.  It'd only be a soft dependency obviously.
>
> Oh, so we'd need to maintain both the LLVM and the traditional expression
> execution code?  That seems like a bit of a pain, but maybe we can live
> with it.

Yea, that's why I converted the "traditional" expression evaluation into
a different format first - that way the duplication is a lot
lower. E.g. scalar var eval looks like:
    EEO_CASE(EEO_INNER_VAR):        {            int attnum = op->d.var.attnum;
            Assert(op->d.var.attnum >= 0);            *op->resnull = innerslot->tts_isnull[attnum];
*op->resvalue= innerslot->tts_values[attnum];            EEO_DISPATCH(op);        }
 

in normal evaluation and like
        case EEO_INNER_VAR:            {                LLVMValueRef value, isnull;                LLVMValueRef
v_attnum;
                v_attnum = LLVMConstInt(LLVMInt32Type(), op->d.var.attnum, false);                value =
LLVMBuildLoad(builder,LLVMBuildGEP(builder, v_innervalues, &v_attnum, 1, ""), "");                isnull =
LLVMBuildLoad(builder,LLVMBuildGEP(builder, v_innernulls, &v_attnum, 1, ""), "");
LLVMBuildStore(builder,value, v_resvaluep);                LLVMBuildStore(builder, isnull, v_resnullp);
 
                LLVMBuildBr(builder, opblocks[i + 1]);                break;            }
for JITed evaluation.


> I'm not entirely thrilled with the idea of this being a configure-time
> decision, because that forces packagers to decide for their entire
> audience whether it's okay to depend on LLVM.  That would be an untenable
> position to put e.g. Red Hat's packagers in: either they screw the people
> who want performance or they screw the people who want security.

Hm. I've a bit of a hard time buying the security argument here. Having
LLVM (not clang!) installed doesn't really change the picture that
much. In either case you can install binaries, and you're very likely
already using some program that does JIT internally. And postgres itself
gives you plenty of ways to execute arbitrary code as superuser.

The argument for not install a c compiler seems to be that it makes it
less convenient to build an executable. I doubt that having a C(++)
library for code generation is convenient enough to change the picture
there.

> I think it'd be all right if we can build this so that the direct
> dependency on LLVM is confined to a separately-packageable extension.
> That way, a packager can produce a core postgresql-server package
> that does not require LLVM, plus a postgresql-llvm package that does,
> and the "no compiler please" crowd simply doesn't install the latter
> package.

That should be possible, but I'm not sure it's worth the effort. The JIT
infrastructure will need resowner integration and such. We can obviously
split things so that part is independent of LLVM, but I'm unconvinced
that the benefit is large enough.


> The alternative would be to produce two independent builds of the
> server, which I suppose might be acceptable but it sure seems like
> a kluge, or at least something that simply wouldn't get done by
> most vendors.

Hm. We could make that a make target ourselves ;)

Regards,

Andres