The existing traditional cryptosystems, such as RSA, DES, IDEA, SAFER and FEAL, are not ideal for image encryption because of their slow speed and ineffectiveness in removing the correlations of the adjacent pixels. Meanwhile chaos-based cryptosystems, which have been extensively used over the past two decades, are almost all based on symmetric cryptography. Symmetric cryptography is much faster than asymmetric ciphers, but the requirements for key exchange make them hard to use. To remedy this imperfection, a hybrid-key based image encryption and authentication scheme is proposed in this paper. In particular, ergodic matrices are utilized not only as public keys throughout the encryption/decryption process, but also as essential parameters in the confusion and diffusion stages. The experimental results, statistical analysis and sensitivity-based tests confirm that, compared to the existing chaos-based cryptosystems, the proposed image encryption scheme provides a more secure means of image encryption and transmission.