Máš to tam napsáno...
column "ar.rules" must appear in the GROUP BY clause
Takže
SELECT ar.rules, ar.size, ar.per_post, COUNT(f.id) FROM attach_2_rules AS ar, attach_2_files AS f,
posts AS p, topics AS t WHERE group_id=4 AND p.id=3 AND t.id=p.topic_id AND ar.forum_id=t.forum_id
GROUP BY f.post_id, ar.rules, ar.size, ar.per_post LIMIT 1;