Mike Hays - Virtualization, PowerShell, and more...

Here’s a very quick Quicktip.  Like most people, I run the vCenter Infrastructure Client (aka VirtualCenter Client, aka VIC) to manage Windows guest OSs.

To make this easier, I change the “scheme” for my mouse pointers from the default to 3D-Bronze in order to make it easier to tell if the VIC console of the guest has “captured” my mouse.  You can see in the video below the mouse cursor change from 3D-Bronze to the default used on all the Windows Server OS images I manage (and likely all the ones you manage as well).

Here’s the procedure to make this simple change:

1. Open the Control Panel on the local system (your workstation).
2. In the Control Panel, double-click on the Mouse applet.
3. In the Mouse Properties applet, click on the Pointers tab.
4. Change the Scheme to 3D-Bronze (or any other scheme were the “Normal Select” pointer isn’t white).
5. Click OK to close the Mouse Properties applet and accept the changes.

As an added bonus, this cursor change occurs for Remote Desktop and Citrix sessions as well.

Happy Virtualizing!

Mike

Anyone who has used PowerShell is familiar with how it shows the current working directory in the prompt as below:

PS C:\Documents and Settings\Mike\PowerShell\Scripts>

This is all well and good until you’re knee deep in directories (or registry keys, or the structure of any another provider) and the prompt is taking up more than half of the current line. Naturally PoSH wraps, but this can make it hard to read what you’ve typed or cut and pasted.

Since the prompt is actually defined by a function named ‘prompt’, a quick and dirty way to shorten the prompt is to type:

function prompt {”>”}

But I usually still want to know where I am.  You can modify your $profile to include a ‘prompt’ function that overrides the default, and do it in such a way that long path descriptions are a thing of the past. Just add the following to your $profile and your long prompts will appear similar to the following:

PS C:\Documents and Settings\…\Scripts>

###
# TruncatePath
# Version 1.1 (14 Apr 2009)
# Description: Replaces long provider paths in the prompt with ellipses
# Notes: Place in your profile
#
# By Mike Hays, <a href="http://www.mike-hays.net">http://www.mike-hays.net</a>
###
 
$MaxPathLength = 32
$ShowFullPath = $False
 
$prompt = New-Object System.Object
$prompt  | Add-Member -type NoteProperty -name MaxPathLength -value $MaxPathlength
$prompt  | Add-Member -type NoteProperty -name ShowFullPath -value $ShowFullPath
 
Function Prompt
{
	$currentPath = (Get-Location).Path
 
	if ( ($currentPath.Length -gt $prompt.MaxPathlength) -and ($prompt.ShowFullPath -ne $true) `
		-and (($currentPath.Split("\")).Count -gt 3) )
	{
		$currentPathSplit = $currentPath.Split("\")
		$truncatedPath = $currentPathSplit[0] + "\" + $currentPathSplit[1] + "\" `
			+ "..." + "\" + $currentPathSplit[$currentPathSplit.Length - 1]
 
		if ($truncatedPath.Length -lt $currentPath.Length)
		{
			$displayPath = $truncatedPath
		}
		else
		{
			$displayPath = get-location
		}
	}
	else
	{
		$displayPath = get-location
	}
 
	Write-Host ("PS ") -NoNewLine -ForegroundColor DarkYellow ; Write-Host ("" + `
	$displayPath + "&gt;") -NoNewLine
	return " "
}

You can get the full path if you need it by typing “Get-Location” or its default alias “pwd” (a unixism for print working directory), or you can disable path truncation by typing:

$prompt.ShowFullPath = $true

Happy PoSHing!

Mike

It is sometimes necessary to cause VMware ESX 3.x to recreate its bootloader.   This is generally done by running the following command sequence:

esxcfg-boot -b

esxcfg-boot -p

esxcfg-boot -r

You might find that these commands run indefinitely on your ESX servers if you alias ‘cp’ or ‘mv’ to ‘cp -i’ or ‘mv -i’ respectively in your /root/.bashrc file.  The ‘-i’ forces the command to run interactively meaning that you are prompted to copy or move the file(s) unless you use ‘-f’ to force the operation.

esxcfg-boot writes and runs scripts to recreate the bootloader, and it uses cp and mv in these scripts (at least it uses one of them, I’m not sure which).  Since it doesn’t include a ‘-f’ in the script, the command appears to hang.

To complete the operation, type ‘y <enter>’ three times, wait ten seconds and type ‘y <enter>’ three times again.  After that, remove the aliases from your /root/.bashrc file to prevent future issues.

Note that you can’t simply ‘unalias cp’ or ‘unalias mv’ prior to running esxcfg-boot, as esxcfg-boot spawns a sub-process to run the script and this process will run /root/.bashrc as part of it’s environment setup.

Thanks to Eric Michaelis, my friend and colleague, for this tip.

Here’s the scenario: You need to move a large number of ESX guest OSs from one datacenter to another.  To make things interesting, there isn’t enough bandwidth on the WAN to move the guest OSs on the wire or replicate the data, so it has to be out of band.  Also, you’re on a tight budget and timeline, so you have to use large commodity USB, Firewire, or eSATA drives to move the guest OSs.  Also, the source ESX servers are a mixture of ESX 2.x and 3.x.

That’s a lot to contend with, but here is one way to do it with limited downtime (1-3 days depending on the data footprint and shipping time).

1. Install vCenter Converter 4.0 Server/Client Standalone on a system that has the required external drive connection (USB, eSATA, etc.).  If you’re using USB, make sure it is USB 2.0.  Even if you know the system is USB 2.0, make sure it is enabled in the system BIOS (HP disables USB 2.0 for some reason on many of their server models that support USB 2.0.)
2. Create a local administrator account in each Windows guest, and make sure you know the root password for any *nix guests.
3. Note the amount of RAM and number of CPUs allocated for each guest.  You’ll need this later when you import the guest, as Converter will probably “truncate” these resources to make the guest “fit” in a running state on your Converter server system.  (<rant>This is annoying and Converter should warn you that it is going to do this, and give you the option to not change the guest OS configuration since it is common to not run the guest on the Converter server itself.</rant>).
4. Optional: Pre-configure the guest with the destination IP address and associated information (gateway, etc.).
5. Shutdown the guest OS.
6. Use Converter to export the guest OS to the USB drive.  Note that you must connect to a VirtualCenter 2.x server if the guest is hosted on ESX 2.x.  I had better luck when I connected directly to ESX 3.x instead of VirtualCenter which timed out periodically.  This might be environmental and your mileage may vary.  Either way, I recommend you tell Converter to export to Workstation 5.0 format, and that you tell it in the second to last wizard page not to modify the guest OS.
7. Repeat steps 2-5 above as needed.
8. Transport the drives to the destination.
9. Reverse the process above, using Converter to import a Workstation guest OS and “converting” it to your ESX server or vCenter server.  Feel free to modify the guest OS by installing the VMware tools or changing it’s memory/CPU allocation as needed.  You will probably need to adjust the number of CPUs and/or RAM in the guest based off what you documented in step 3 above.

Note that Converter only pulls the actually data out of the vmdk file.  For example, if you have a 200 GB vmdk  file with only 4 GB of data, Converter will only pull 4 GB of data across the network, and only use 4 GB of space on your external storage device.  Even so, you will likely see transfer rates of only 0.5GB-1GB/minute, so expect several hours to days of downtime if you are moving guests with tens to hundreds of gigabytes of data.

I tried several ways to find and eliminate any bottlenecks and increase Converter’s throughput.  But the throughput was about the same whether I used an external USB 2.0 disk or local internal disk, whether I used 100Mb/Full or 1Gb Ethernet, or whether I used a system with multiple CPUs/cores or not (although an old laptop I tried initially did cause performance issues).  The bottleneck appears to be in Converter itself and I’ve read of others who experienced similar throughput using it.

Approaches that might improve throughput, but create other issues:

Use vmkfstools: Using vmkfstools is faster, but copying the exported disks onto the USB drive erases most of the time saved.  You can connect the USB drive directly to an ESX server, but the Service Console is rather slow when accessing USB, even USB 2.0.  Also, since vmkfstools is a vmdk disk tool, you are exporting the disks and not the guest itself.  So you have an vmx file you need to copy as well.  This might be acceptable, and if all your disks are on the same volume and you’re running ESX 3.x you can even get away with copying the vmx file unmodified.  But if your disks are on different LUNs, or your ESX server is 2.x, you might find yourself spending more time in vmx files than any sane person should.  I’m all about scripting, and you can definitely script changes to vmx files, but depending on the environment, there can be a large number of variables for which you have to account.

Naturally you can recreate the vmx file from scratch using your favorite approach to creating a new guest OS.  But this can be time consuming if you are moving a large number of guests.

Use cp: This is generally a bad idea.  The ‘cp’ command takes 1MB nibbles of vmdk files which generates a large number of LUN locks.  Also, at least on ESX 2.x, using ‘cp’ to copy a vmdk file to a non-VMFS partition would corrupt the vmdk file.

Install Converter server in the ESX Service Console (temporarily): It can be done and reducing the number of hops seems like a good idea if you don’t mind rebuilding an ESX server when your done, but when I attempted this approach it actually doubled my export times.  As I stated above, the Service Console reportedly doesn’t handle USB 2.0 very well, but I think Service Console memory was actually my bottleneck, and you can’t give the Service Console more than 800 MB of RAM.  If you attempt this approach, note that you will need to change the HTTP and HTTPS TCP ports during the Converter server installation, and you will need to use a remote Converter client.

Use an NFS device you can transport: Now you’re talking.  If you are using ESX 3.x and have an NFS device with enough disk space that you can transport, you could use this option, and it would be faster.  Instead of using Converter, you can either clone or cold migrate the guests.  And if the NFS device can perform well enough, you could even reduce downtime by using storage migration to move running guests to the NFS device (Note: As of this writing VMware doesn’t support hot storage migrations where NFS is concerned, but it reportedly works fine in most cases).  Naturally you will need an NFS device you can move, and this device may require special consideration when shipping.

Use a SAN you can transport: This too would be faster and will even work with ESX 2.x, but you’ll need an extra SAN, and I didn’t have one of these laying around.

Replicate the data: Now your breaking the rules. This is probably the ideal solution, but it’s not out-of-band.  You can replicate the LUNs, make them visible to your ESX server, register the guests, and go.  VMFS2 LUNs will need an upgrade to VMFS3 if you are exclusively using ESX 3.x at your destination.  You can also use vmkfstools to export the guests vmdk files and copy them over the WAN, importing them on the other end.  Note that VMware doesn’t support using Converter over a WAN–expect timeouts if you attempt to do so.

Rebuild the guest OS from scratch: In some cases this is actually a good option.  For example, if there is network connectivity between the sites (just not enough to replicate gigabytes of data), and you have a Windows Active Directory domain controller you need to “move” it is probably cleaner to build the server fresh at the destination.  This might apply to other infrastructure servers such as DHCP, WINS, or DNS servers as well.

The process above is the best I’ve been able to find for out-of-band relocations of guest OSs.  Please give any feedback if you have found any better approaches that meet the requirements of the scenario or have other ideas to make this type of migration more pleasant.

Good luck, and happy virtualizing!

Of the execution policies allowed by PowerShell, RemoteSigned provides the greatest degree of flexibility while still providing a measure of security.  But the implications of RemoteSigned are not well explained by the PowerShell documentation.  I will endeavor to explain this setting fully here.

Downloaded Files

Windows XP SP2 and later and Windows Server 2003 SP1 added the Attachment Execution Service.  This API makes it easier for applications to write security zone information into an NTFS Alternate Data Stream (ADS) on the file.  An Alternate Data Streams is a pseudo-hidden metadata space that can contain standard Unicode characters.  ADSs are not widely used in general.  When invoked, the Attachment Execution Service writes the file’s originating Security Zone to the Zone.Identifier ADS on the file.  For example, a file downloaded from the Internet using Internet Explorer or Firefox 3 will have the following information in the file’s Zone.Identifier ADS:

[ZoneTransfer]
ZoneId=3

You can see this information by opening the Zone.Identifier ADS for a file in Notepad as follows:

notepad c:\folder\filename.ps1:Zone.Identifier

If you are prompted to create the “file”, no Zone.Identifier ADS exists.

PowerShell uses the information written to the Zone.Identifier ADS to determine whether to run a script under the RemoteSigned execution policy.  If a script is marked as originating in the Internet or Untrusted security zones, PowerShell will abort execution and display an error similar to the following:

File C:\folder\script.ps1 cannot be loaded. The file C:\folder\script.ps1
is not digitally signed. The script will not execute on the system. Please see
“get-help about_signing” for more details..
At line:1 char:2
+ &  <<<< “C:\folder\script.ps1″

If the Zone.Identifier ADS doesn’t exist, or if it shows any ZoneId besides 3 for Internet or 4 for Untrusted, the script will execute.

The table below summarizes how PowerShell handles each ZoneId when a script runs under the RemoteSigned execution policy:

Various applications use the Attachment Execution Service to set a Zone.Identifier ADS to different values.  The following lists a few examples:

Note that files that are compressed when downloaded and then extracted typically inherit the Zone.Identifier ADS of the compressed “parent”.

If you download a PowerShell script with an application that writes the Zone.Identifier ADS ZoneId to 3 or 4, and your PowerShell execution policy is RemoteSigned, you have a couple of options to get the script to execute:

1. Right click on the file, select Properties, and click the Unblock button.  This will delete the Zone.Identifier ADS.
2. Open the file and copy the contents to a new file in Notepad or your favorite editor.  Then save the new file.  The new file won’t have a Zone.Identifier ADS.
3. Open the Zone.Identifier ADS directly in Notepad and change the ZoneId (Note: You will need to close PowerShell and reopen it if you already attempted to run the script, as it seems to cache the values in the Zone.Identifier ADS, and changing it via Notepad doesn’t trigger PowerShell to recheck it).
4. Run the following command to blast the Zone.Identifier ADS:
   echo $null > ./script.ps1:Zone.Identifier
5. Run a script that is signed by the publisher and indicate that you trust the publisher when prompted.
6. Sign the script using the Set-AuthenticodeSignature cmdlet.  But you should only do this if you or your organization wrote the script or you did a strict code review on the script.  By signing the file you are saying to the world “My organization and I mark this script as safe and take responsibility for it”.  You shouldn’t sign someone else’s script and release it back into the wild.
7. Change the PowerShell execution policy to Unrestricted (Note: I place this option in my “Bad Ideas” category.  I’ll explain why I think so in a future post).

Naturally you should only run scripts that you have reviewed and deemed trustworthy.

While the Zone.Identifier ADS is a feature that can improve the security of your environment, it has limitations.

• Files without a Zone.Identifier ADS are trusted by default.
• Anyone with write access to a file can modify the Zone.Identifier ADS contents or delete it altogether.

Even with these limitations, using RemoteSigned reduces the likelihood that a worm or a malicious website can use PowerShell to attack your systems.

File Shares and Security Zones

While we usually think of security zones in relation to Internet Explorer, these zones also apply to mapped drives and UNC paths.  PowerShell considers the security zone in which the file currently resides and applies the execution policy as defined.  By default Windows XP and Vista consider UNC paths part of the Local Intranet, and will allow the execution of scripts on these paths with the RemoteSigned execution policy.  Windows Server 2003 (Win2k3) and Windows Server 2008 (Win2k8) on the other hand run under a higher security standard, and by default consider UNC paths to be in the “Internet” Security Zone, even if they are on the local network.  Typically this behavior is desirable, as servers should by their very nature run in a more restricted fashion.

If a file exists on a remote file share in the Internet or Untrusted security zones, and your PowerShell execution policy is set to RemoteSigned you can do the following to get the script to run:

1. Copy the file locally.  Unless the file already has a Zone.Identifier ADS, one will not be added.
2. Run a script that is signed by the publisher and indicate that you trust the publisher when prompted.
3. Sign the script.  The same guidance on this process from above still applies.
4. Add the server in the UNC path to the Intranet or Trusted security zones (Use Internet Explorer to configure this setting.  Add the path as follows: file://servername)
5. Change the PowerShell execution policy to Unrestricted (Still a bad idea).

Once again, you should only do this for scripts and paths that you trust.

Be aware that that most restrictive of the Security Zone and Zone.Identifier ADS will apply to the scripts that you run.  For example, a script that exists on a trusted file share won’t run if it’s Zone.Identifier ADS marks the file as untrusted.

Using the RemoteSigned Execution Policy

Because it reduces the likelihood that PowerShell can be used by a worm or malicious website to attack your systems, I recommend that you consider the RemoteSigned execution policy the minimum setting for any system.   Since it strikes a good balance between usability and security, I recommend this execution policy for servers that aren’t in high security areas or that aren’t intended for non-admin users such as Citrix or Terminal Servers.  I also recommend this setting for desktops used to develop PowerShell scripts or that are used by power users that need to create their own PowerShell scripts.

I don’t recommend this setting as the default for general purpose desktops in your organization.  For these systems I recommend AllSigned which I will cover in detail in a future blog post.

Conclusion

By allowing us to control the scripts that PowerShell can execute without interaction, Microsoft has made a effort to prevent PowerShell from becoming a target for worms and black hat hackers.

While it might be tempting to set the PowerShell execution policy to Unrestricted, RemoteSigned does a better job of preventing the execution of malicious scripts.  Also, since locally created scripts are allowed to run, scripters will find they are unaffected by this setting.  And since scripts that are downloaded from a website can be easily “unblocked”, there is no reason to run completely unprotected (Especially since most scripters I know don’t download PowerShell scripts, but rather copy the text of the script from a website and paste it into a local file).

The setting AllSigned is recommended for all other systems that need to run scripts (necessitating the signing of these scripts), and the default Restricted should be maintained for systems that should never execute PowerShell scripts.

You need to give careful consideration for the different PowerShell execution policies, and how you decide to implement them in your organization.

Happy PoSHing,

Mike Hays

This is my first blog post since I switched hosting providers and changed my blog to WordPress.  I’m really happy with my new hosting provider (www.3essentials.com) and WordPress is excellent.

On this blog I will discuss what I have learned about:

• Virtualization with a focused on VMware ESX and the other parts of VMware’s Virtual Infrastructure
• Microsoft PowerShell, one of the most exciting and, well, powerful things to come out of Redmond in a long time
• And other technologies and ideas that I think are interesting

You can learn more about me by visiting my “regular” website at www.mike-hays.net.  You can contact me directly by visiting www.mike-hays.net/contact.html

I look forward to reading your comments.

Mike Hays