"If you declared the function with output parameters, write just RETURN NEXT with no expression. On each execution, the current values of the output parameter variable(s) will be saved for eventual return as a row of the result. Note that you must declare the function as returning SETOF record when there are multiple output parameters, or SETOF sometype when there is just one output parameter of type sometype, in order to create a set-returning function with output parameters."
Either:
CREATE OR REPLACE FUNCTION words_check_words(
IN in_uid integer,
IN in_gid integer,
IN in_tiles jsonb
OUT out_word varchar,
OUT out_score integer
) RETURNS SETOF RECORD AS
$func$
Or:
CREATE OR REPLACE FUNCTION words_check_words(
IN in_uid integer,
IN in_gid integer,
IN in_tiles jsonb
) RETURNS TABLE (out_word varchar, out_score integer) AS
$func$
And then I assign values to the variables and call RETURN NEXT: