Face image generation and animation have been a longstanding task. Although many 2D generative models yield excellent manipulations in 2D space, they often suffer from 3D inconsistency and undesirable artifacts when rendering from different camera viewpoints, and thus are not suitable for animations in video. Recently, 3D-aware GANs extend 2D GANs by using underlying 3D representations. Although these methods can preserve the 3D consistency across different viewpoints, they cannot achieve fine-grained control over attributes, most importantly, facial expression. In this paper, we propose an animatable 3D-aware face image generation method. Our framework mainly consists of a template implicit field and a 3D deformation field. The template field represents the canonical space and is shared across the same identity. Different expressions can be generated by deforming the manifolds in the target space to the canonical space. We enforce the generation to follow a prior 3D face parametric model by incorporating 3D-level imitative learning to encourage the deformation field to follow 3D prioris. Experiments show our method can produce high-quality animatable video avatars with strong visual 3D consistency.