Oracle 10g is not supposed to be installed on AIX 5.3 but you can try...
runInstaller -ignoreSysPrereqs