Thread: FP16 Support?
Hello, How about your thought for support of half-precision floating point, FP16 in short? https://en.wikipedia.org/wiki/Half-precision_floating-point_format Probably, it does not make sense for most of our known workloads. Our supported hardware platform does not support FP16 operations, thus, it will be executed by FP32 logic instead. On the other hands, folks of machine-learning said FP32 values provides too much accuracy than necessity, and FP16 can pull out twice calculation throughput than FP32. In fact, recent GPU models begin to support FP16 operations by the wired logic. https://en.wikipedia.org/wiki/Pascal_(microarchitecture) People often make inquiries about management of the data-set to be processed for machine-learning. As literal, DBMS is software for database management. There are some advantages, like flexible selection of parent population, pre- or post-processing of the data-set (some algorithms requires to normalize the input data in [0.0 - 1.0] range), and so on. If we allow to calculate/manipulate/store the FP16 data in binary compatible form, it is much efficient way to fetch binary data for machine-learning engines. No special operations for machine-learning are needed, but usual arithmetic operations, type cast, array operations will be useful, even though it internally uses FP32 hardware operations on CPUs. Any opinions? Thanks, -- HeteroDB, Inc / The PG-Strom Project KaiGai Kohei <kaigai@heterodb.com>
Kohei KaiGai <kaigai@heterodb.com> writes:
> How about your thought for support of half-precision floating point,
> FP16 in short?
This sounds like a whole lotta work for little if any gain. There's not
going to be any useful performance gain from using half-width floats
except in an environment where it's the individual FLOPs that dominate
your costs. PG is not designed for that sort of high-throughput
number-crunching, and it's not likely to get there anytime soon.
When we can show real workloads where float32 ops are actually the
dominant time sink, it would be appropriate to think about whether
float16 is a useful solution. I don't deny that we could get there
someday, but I think putting in float16 now would be a fine example
of having your priorities reversed.
regards, tom lane
On 2017-11-13 20:21:47 -0500, Tom Lane wrote: > Kohei KaiGai <kaigai@heterodb.com> writes: > > How about your thought for support of half-precision floating point, > > FP16 in short? > > This sounds like a whole lotta work for little if any gain. There's not > going to be any useful performance gain from using half-width floats > except in an environment where it's the individual FLOPs that dominate > your costs. PG is not designed for that sort of high-throughput > number-crunching, and it's not likely to get there anytime soon. > > When we can show real workloads where float32 ops are actually the > dominant time sink, it would be appropriate to think about whether > float16 is a useful solution. I don't deny that we could get there > someday, but I think putting in float16 now would be a fine example > of having your priorities reversed. Agree that there's no performance argument. I think you could kinda sorta make an argument for higher storage density in cases where a lot of floats are stored in the database. I'd personally still consider that not worthwhile to invest time in, but ... Greetings, Andres Freund
On Tue, Nov 14, 2017 at 1:11 PM, Kohei KaiGai <kaigai@heterodb.com> wrote: > Any opinions? The only reason I can think of for having it in core is that you might want to use standard SQL notation FLOAT(10) to refer to it. Right now our parser converts that to float4 but it could map precisions up to 10 to float2. The need for such special treatment is one of my arguments for considering SQL:2016 DECFLOAT(n) in core PostgreSQL. But this case is different: FLOAT(10) already works, it just maps to a type with a larger significand, as permitted by the standard. So why not just do these short floats as an extension type? -- Thomas Munro http://www.enterprisedb.com
2017-11-14 10:21 GMT+09:00 Tom Lane <tgl@sss.pgh.pa.us>: > Kohei KaiGai <kaigai@heterodb.com> writes: >> How about your thought for support of half-precision floating point, >> FP16 in short? > > This sounds like a whole lotta work for little if any gain. There's not > going to be any useful performance gain from using half-width floats > except in an environment where it's the individual FLOPs that dominate > your costs. PG is not designed for that sort of high-throughput > number-crunching, and it's not likely to get there anytime soon. > > When we can show real workloads where float32 ops are actually the > dominant time sink, it would be appropriate to think about whether > float16 is a useful solution. I don't deny that we could get there > someday, but I think putting in float16 now would be a fine example > of having your priorities reversed. > A typical workload I expect is, a data-scientist stores more than million records which contains feature vectors hundreds or thousands dimension. It consumes storage space according to the width and number of items, and it also affects to the scan performance. In addition, it may need extra data format conversion if user's script wants to process the feature vector as half-precision floating point. If a record contains a feature vector with 1000 dimension by FP32, a million records consume 4GB storage space. In case when FP16 is sufficient, it consumes only half of space, thus, it takes half of time to export data arrays from the database. Of course, our own extension can define own "half" or "float2" data types regardless of the core feature, however, I thought it is a useful feature for other people also. Thanks, -- HeteroDB, Inc / The PG-Strom Project KaiGai Kohei <kaigai@heterodb.com>
2017-11-14 10:32 GMT+09:00 Andres Freund <andres@anarazel.de>: > On 2017-11-13 20:21:47 -0500, Tom Lane wrote: >> Kohei KaiGai <kaigai@heterodb.com> writes: >> > How about your thought for support of half-precision floating point, >> > FP16 in short? >> >> This sounds like a whole lotta work for little if any gain. There's not >> going to be any useful performance gain from using half-width floats >> except in an environment where it's the individual FLOPs that dominate >> your costs. PG is not designed for that sort of high-throughput >> number-crunching, and it's not likely to get there anytime soon. >> >> When we can show real workloads where float32 ops are actually the >> dominant time sink, it would be appropriate to think about whether >> float16 is a useful solution. I don't deny that we could get there >> someday, but I think putting in float16 now would be a fine example >> of having your priorities reversed. > > Agree that there's no performance argument. I think you could kinda > sorta make an argument for higher storage density in cases where a lot > of floats are stored in the database. I'd personally still consider > that not worthwhile to invest time in, but ... > Yes. I don't think PostgreSQL will perform massive calculation intensive workloads by itself. (If people want, we have PL/CUDA to run advanced algorithms in UDF, as an aside.) However, data management is a top priority job for DBMS, and storage density directly affects to the performance to load data stream onto multicore processors (like GPUs). My proposition intends to reduce storage/memory consumption when user stores massive amount of floating point arrays database. Thanks, -- HeteroDB, Inc / The PG-Strom Project KaiGai Kohei <kaigai@heterodb.com>
2017-11-14 10:33 GMT+09:00 Thomas Munro <thomas.munro@enterprisedb.com>: > On Tue, Nov 14, 2017 at 1:11 PM, Kohei KaiGai <kaigai@heterodb.com> wrote: >> Any opinions? > > The only reason I can think of for having it in core is that you might > want to use standard SQL notation FLOAT(10) to refer to it. Right now > our parser converts that to float4 but it could map precisions up to > 10 to float2. The need for such special treatment is one of my > arguments for considering SQL:2016 DECFLOAT(n) in core PostgreSQL. > But this case is different: FLOAT(10) already works, it just maps to a > type with a larger significand, as permitted by the standard. So why > not just do these short floats as an extension type? > Our extension will be able to provide its own "half" or "float2" data type using CREATE TYPE, indeed. I thought it is useful to other people, even if they are not interested in the in-database analytics with GPU, to reduce amount of storage consumption. Of course, it is my opinion. Thanks, -- HeteroDB, Inc / The PG-Strom Project KaiGai Kohei <kaigai@heterodb.com>
On Tue, Nov 14, 2017 at 6:49 PM, Kohei KaiGai <kaigai@heterodb.com> wrote: > 2017-11-14 10:33 GMT+09:00 Thomas Munro <thomas.munro@enterprisedb.com>: >> On Tue, Nov 14, 2017 at 1:11 PM, Kohei KaiGai <kaigai@heterodb.com> wrote: >>> Any opinions? >> >> The only reason I can think of for having it in core is that you might >> want to use standard SQL notation FLOAT(10) to refer to it. Right now >> our parser converts that to float4 but it could map precisions up to >> 10 to float2. The need for such special treatment is one of my >> arguments for considering SQL:2016 DECFLOAT(n) in core PostgreSQL. >> But this case is different: FLOAT(10) already works, it just maps to a >> type with a larger significand, as permitted by the standard. So why >> not just do these short floats as an extension type? >> > Our extension will be able to provide its own "half" or "float2" data type > using CREATE TYPE, indeed. I thought it is useful to other people, even > if they are not interested in the in-database analytics with GPU, to reduce > amount of storage consumption. > > Of course, it is my opinion. Perhaps what the world needs is a single extension called ieee754 that would provide binary16 AND decimal32, decimal64. Seems a bit unlikely to happen though, because even though that's a single standard the people who care about binary floats and the people who care about decimal floats go to different conferences and the other base isn't on their radar (joking, sort of). I also wonder if there could be some way to make it so that the FLOAT(n) and DECFLOAT(n) typename conversion happens somewhere outside the parser so that extensions can participate in that kind of trick... -- Thomas Munro http://www.enterprisedb.com
Just for your information. I tried to implement "float2" according to IEEE 754 specification, as a custom data type of PG-Strom. https://github.com/heterodb/pg-strom/blob/master/src/float2.c https://github.com/heterodb/pg-strom/blob/master/sql/float2.sql The recent GPU device (Maxwell or later) supports "half" precision data type by the hardware, so it might be valuable for somebody. Thanks, 2017-11-14 14:49 GMT+09:00 Kohei KaiGai <kaigai@heterodb.com>: > 2017-11-14 10:33 GMT+09:00 Thomas Munro <thomas.munro@enterprisedb.com>: >> On Tue, Nov 14, 2017 at 1:11 PM, Kohei KaiGai <kaigai@heterodb.com> wrote: >>> Any opinions? >> >> The only reason I can think of for having it in core is that you might >> want to use standard SQL notation FLOAT(10) to refer to it. Right now >> our parser converts that to float4 but it could map precisions up to >> 10 to float2. The need for such special treatment is one of my >> arguments for considering SQL:2016 DECFLOAT(n) in core PostgreSQL. >> But this case is different: FLOAT(10) already works, it just maps to a >> type with a larger significand, as permitted by the standard. So why >> not just do these short floats as an extension type? >> > Our extension will be able to provide its own "half" or "float2" data type > using CREATE TYPE, indeed. I thought it is useful to other people, even > if they are not interested in the in-database analytics with GPU, to reduce > amount of storage consumption. > > Of course, it is my opinion. > > Thanks, > -- > HeteroDB, Inc / The PG-Strom Project > KaiGai Kohei <kaigai@heterodb.com> -- HeteroDB, Inc / The PG-Strom Project KaiGai Kohei <kaigai@heterodb.com>