I can reproduce this on master. The same happens with all the skip_nulls paths, such as jsonb_object_agg_strict(), and JSON_OBJECT() and JSON_OBJECTAGG() using ABSENT ON NULL RETURNING jsonb. The jsonb array versions keep the value, jsonb_agg_strict('null'::jsonb) returns [null].
Is dropping JSON nulls here intended? I looked at the old threads where skip_nulls was discussed [1][2], they are only about SQL NULLs. The docs just say "null".