There is no smiley size restrictions (at least not in nuke 7) for image size or file size.

I know that you uploaded the smileys but did you actually activate them in your smiley admin panel?

You have to go to your phpBB admin panel (Forums Admin), look for "General Admin." and then click on "Smilies". You will now see a list of currently active smilies in your forums. Click on "Add a New Smiley", create an easy to remember shortcut code such as :-0: :winky: or something that represents the smiley that your activating, locate the smiley image file from your smiley drop down box and lastly, give the smiley a description.

Click submit and you should now see your new smiley on the smiley list.