Here is the output and performance improvement:
postgres=# \timing on
Timing is on.
postgres=# DROP MATERIALIZED VIEW IF EXISTS s CASCADE;
NOTICE: materialized view "s" does not exist, skipping
DROP MATERIALIZED VIEW
Time: 0.858 ms
postgres=#
postgres=# CREATE MATERIALIZED VIEW s AS SELECT generate_series as x, null as y FROM generate_series(1, 1000000);
SELECT 1000000
Time: 1076.254 ms (00:01.076)
postgres=#
postgres=# CREATE UNIQUE INDEX ON s(x);
CREATE INDEX
Time: 375.026 ms
postgres=# REFRESH MATERIALIZED VIEW CONCURRENTLY s;
REFRESH MATERIALIZED VIEW
Time: 3807.143 ms (00:03.807)
postgres=# CREATE UNIQUE INDEX ON s(y);
CREATE INDEX
Time: 331.382 ms
postgres=# REFRESH MATERIALIZED VIEW CONCURRENTLY s;
REFRESH MATERIALIZED VIEW
Time: 3636.049 ms (00:03.636)postgres=#
As we can see the REFRESH MATERIALIZED VIEW CONCURRENTLY now takes 3636.049 ms
