FileVault (or File Vault) is the cryptographic file system developed by Apple and introduced with MacOS 10.3.
FileVault works by storing each user's home directory in an encrypted ".sparseimage" file. The file is automatically mounted when the user logs in and unmounted when the user logs out. All of the user's files and preferences are stored in this file. The file's encryption key is stored in the .sparseimage file, but that encryption key is itself encrypted with the user's login password.
There are no known attacks against File Vault other than a brute force attack on the user's password.
As part of the security enhancements in OS X 10.5 (Leopard) Apple have moved from AES-128 to AES-256 for the encryption used in the disk image.
In Mac OS X 10.7 (Lion) a full volume encryption variant was introduced.