Continuing with the news of vSphere 7, this time we will see the security enhancements in two different posts.
Virtual Machine Encryption Enhancements
Improvements to VM Encryption Functionality
vSphere 7 includes several enhancements to the VM encryption feature that was introduced in vSphere 6.5:
• Some VM operations now support encrypted VMs.
• With vSphere HA enabled and at least one encrypted VM, encryption mode is enabled on all hosts in the cluster.
• New APIs improve the management of the VM encryption environment.
• New events and alarms monitor the VM encryption environment.
These enhancements improve the general functionality of VM encryption. They are based on competitive analysis and feedback from VMware customers and partners.
Encryption Use Cases
By enabling encryption on VMs, we can protect our confidential data:
• VMDKs are encrypted and can be accessed only with a digital key.
• vSphere access control restricts access to the digital key for decryption.
• In-flight data is encrypted.
To mitigate the risk of an administrator accessing data without authorization, we can encrypt VMs and provide access to encrypt and decrypt to a subset of administrators. To provide this access, we use the built-in cryptographic privileges in vSphere.
A key management server (KMS) generates the digital key for encryption and decryption.
VM Encryption Operations During VM Cloning
In vSphere 7, cryptographic operations are supported during the cloning of a VM or the creation of a VM from a template.
When cloning an unencrypted VM, we can encrypt the destination VM by changing the storage policy of the destination VM to the VM encryption policy (or custom encryption policy). When cloning an encrypted VM, we can decrypt the destination unencrypted VM by changing the storage policy of the destination VM to a nonencryption policy. When cloning an encrypted VM, we recrypt the destination encrypted VM by using PowerCLI to rekey VMs. We can perform either a shallow or a deep rekey.
A shallow rekey (or recrypt) changes the key encryption key (KEK) associated with the VM. The data encryption key (DEK) is wrapped with the new KEK.
Deep rekey (or recrypt) changes the DEK and recrypts all data using the new DEK. This function requires temporary disk space and is slower than a shallow rekey because all data must be rewritten.
Encryption or Decrypting the Destination VM
To encrypt or decrypt the destination VM, change the VM’s storage policy.
Decrypting a VM can be a security risk. Therefore, as a best practice, restrict most users from the privilege to decrypt VMs.
System Requirements for Encryption, Decryption and Recryption
For encrypting, decrypting, and recrypting VMs during cloning, the following requirements must be met:
• A key management server (KMS) server must be configured in vCenter Server.
• If cloning occurs across ESXi hosts, both source and destination ESXi hosts must be vSphere 7.
Cross vCenter Cloning of Encrypted VMs
In vSphere 7, we can perform cryptographic operations across vCenter Server instances. When cloning an unencrypted VM across vCenter Server instances, we can encrypt the destination VM. When cloning an encrypted VM across vCenter Server instances, we can decrypt or recrypt the destination VM.
VMs can be powered on or powered off when cloning across vCenter Server instances.
Cross vCenter Migration of Encrypted VMs
For migrations, certain rules apply:
• We can migrate an encrypted VM across vCenter Server instances.
• We can´t decrypt the VM during the migration.
• We can´t recrypt the VM during the migration.
VMs can be powered on or powered off when migrating across vCenter Server instances. During the migration operation, we can´t change keys and, therefore, recrypting the VM during the migration is not supported.
We also can´t change the storage policy, which means that decrypting the VM during the migration is not supported.
Architecture for Cross vCenter Server Migrations and Clones
Source and destination vCenter Server instances must be connected to the same KMS cluster. Encrypted VMs, with keys in the shared KMS cluster, can be migrated or cloned between vCenter Server instances. Both source and destination ESXi hosts must be vSphere 7.
Encrypted VMs, with keys in a dedicated KMS cluster, cannot be migrated or cloned to another vCenter Server instance.
VM keys must reside in the shared key management server (KMS) cluster for VM migrations and cloning to occur between vCenter Server instances.
If the destination host is not in encryption mode, we must verify that the appropriate KMS is also available to the vCenter Server instance that manages the destination host. We can set the default KMS cluster to the shared KMS cluster or a different KMS cluster.
Shallow Recrypts of VMs with Snapshots
In vSphere 7, we can perform a shallow rekey of an encrypted VM with snapshots while the VM is powered on or off. We can encrypt only VMs with a single snapshot branch (multiple branches are not supported).
In versions earlier than vSphere 7, we can perform a shallow rekey (recrypt) of an encrypted VM without snapshots while the VM is powered on or off. A deep recryption replaces all keys in a VM with new keys, from top to bottom. This operation requires rewriting all encrypted data in the VM. A shallow recryption replaces only the top-level managed keys in a VM, namely, the VMDiskKey. Only a small amount of encrypted data must be rewritten. As in vSphere 6.7, you use PowerCLI to perform the shallow rekey.
If the recryption operation fails before all snapshots are recrypted in the chain, some snapshots are encrypted with the new key and other snapshots use the old key. To fix this problem, we must perform the shallow recryption again.
Encrypted VMs with Instant Clone Technology
In vSphere 7, Instant Clone Technology supports cloning an encrypted VM (the parent VM). We can create an instant clone of a VM by using PowerCLI or API calls. By default, the encrypted instant clone VM inherits the same keys as the parent VM.
This cloning process has limitations:
• During the instant clone operation, the encrypted VM cannot be recrypted or decrypted.
• After we create an instant clone of an encrypted VM, we can´t recrypt (shallow or deep) or decrypt the source VM or the destination VM.
We can´t change the storage policy or change encryption keys during an instant clone operation.
Pushing Keys to Hosts
In versions before vSphere 7, a cryptographic operation on an ESXi host works like this:
• The host must be enabled for encryption.
• When one host in a cluster enters crypto-safe mode, all the other hosts in the cluster must enter crypto-safe mode.
• vCenter Server retrieves the necessary keys for all hosts in the cluster.
In vSphere 7, a cryptographic operation works like this:
• If vSphere HA is enabled on a cluster, with at least one encrypted VM on a host in the cluster, vCenter Server pushes the necessary VM keys to all hosts in the cluster.
• If vSphere HA is not enabled, vCenter Server pushes VM keys only to the ESXi hosts that VM and its disks are registered on.
vSphere DRS does not affect the cryptographic operation. Only vSphere HA is checked.
VM Encryption Events and Alarms
vSphere 7 logs warnings and errors that provide information about the running state of vCenter Server and the VM encryption environment:
• Warning event:
— Encrypted core dump exists.
• Critical events:
— Insufficient disk space to perform the cryptographic operation
— Key creation failed on the KMS cluster.
— KMS server is unreachable.
These events help us find possible issues.
Warning: Encrypted Core Dump Exists
The encrypted core-dump event is generated whenever an encrypted hostd, vpxa, or application core dump is detected.
The encrypted core-dump event is useful in the following situations:
• We need a core dump to troubleshoot a problem: The event provides the name and location of the core-dump file.
• We perform periodic maintenance and want to free up disk space: The event identifies a core dump, and we can delete the core dump if it is not needed.
Log entries are written to the vCenter logs. For an encrypted hostd core dump, a log entry is written to the hostd logs on the ESXi host. /var/log/hostd.log.
As with vSphere 6.x, you can decrypt or recrypt an encrypted core dump on your ESXi host by using the crypto-util CLI.
vCenter Server generates and captures warning events when an encrypted hostd core dump exists. The message is written to the hostd logs.
Critical Event: Insufficient Disk Space
A critical event is posted when encryption fails because of insufficient disk space. If disk space is low, and you try to encrypt or decrypt a powered-off VM, this event might occur. Twice the amount of disk space is required temporarily to complete encrypting or decrypting operations. We must locate the datastore where the encrypted VM resides and monitor its disk space.
Critical Event: Key Generation Failed
Search for Key creation failed in the /var/log/vmware/vpxd/vpxd.log file. Verify that the KMS is powered on and operating properly. Verify that the KMS and vCenter Server can communicate with each other.
Critical Event: KMS Unreachable
The KMS Unreachable critical event is posted when vCenter Server detects that its connection with the KMS is down. If the connection is down, the KMS health status alarm is triggered.
If the KMS health status alarm is triggered, take the following actions:
• Check vCenter Server logs for related messages.
• Verify that the KMS server is not down or powered off.
• Check the connection between the KMS server and vCenter Server:
— If a proxy is required to reach your KMS, check that it is still reachable.
— Verify that you are using the correct IP addresses or FQDNs.
Closing Note
I hope it has been useful to you. In the next post we will see the second part. See you!