Pagination with hasMany association cakePHP

I have two tables:

Contestant and Votes
Contestant hasMany Votes

I've tried doing a count(Vote.id) as Votes so I can place it on the
recordset and just paginate them but I have no idea where to place it.
I did it on the fields array but it gave me the total count of votes
regardless of the contestant they belong to.

The votes are linked together in the Contestant recordset so what I
did was on my view I did a count($contestant[Vote]) but this can't be
paginated and my client wants to be able to sort the contestants by
votes.

Is there a way I can do something like this on my view?:

sort('Votes', 'count(Vote)'); ?>

Or do I have to create a query which does a count for all the votes
where Contestant.id = Votes.contestant_id ?

Controller Contestant:

function index() {
            $page = 'Contestants';
            $this->set('page', $page);
            $this->paginate =
            array(
                    'order' => 'id ASC',
                    'contain' => array(
                            'Vote' => array(
                                    'fields' => array("Vote.contestant_id",'Vote.id')
                            )
                    )
            $conditions ["Contestant.active"] = 1;
            $this->set('contestants', $this->paginate('Contestant',

$conditions));
}