Who knows which users are going to be in that list???
It doesn't matter. Worse case scenario there is only one user in the result and so all 50 rows are their earliest 50 rows. The system will thus never need more than the earliest 50 rows per user to answer this question.