Yesterday, I made a few mistakes and learned a few lessons.  I was at a client site helping them move to VMware.  We decided to switch one of their virtual machines from one VMware ESXi host to another.  They have shared iSCSI storage (EqualLogic SAN), so it should have been a relatively simple procedure.  I made the mistake of not taking a snapshot of the volume before the switch.  While making the switch, something happened to the volume (not sure if it was accidentally removed or if it was some kind of corruption).  We could detect the iSCSI target but it wasn’t showing in the storage list.  When we went to add storage it showed the volume as blank.  The web interface on the SAN still showed the data there, so we knew (hoped) it wasn’t gone.  After a bit of searching I found this forum thread that hints at a possible fix but lists the solution as a call to VMware support.  We called VMware and successfully used this process to restore the volume.

Via console or SSH access:

login as: root
root@XXX.XXX.XXX.XXX.XXX’s password:

Tech Support Mode successfully accessed.
The time and date of this access have been sent to the system logs.

WARNING – Tech Support Mode is not supported unless used in
consultation with VMware Tech Support.

~ # cd /tmp
/tmp # fdisk -l

Disk /dev/disks/vmhba32:1:0:0: 549.7 GB, 549763153920 bytes
64 heads, 32 sectors/track, 524295 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Device Boot    Start       End    Blocks   Id  System

Disk /dev/disks/vmhba32:0:0:0: 549.7 GB, 549763153920 bytes
64 heads, 32 sectors/track, 524295 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Partition table entries are not in disk order
/tmp # dd if=/dev/disks/vmhba32:0:0:0 of=/tmp/ddout bs=1M count=2
2+0 records in
2+0 records out
/tmp # hexdump -C ddout |less
/tmp # fdisk -l /dev/disks/vmhba32:0:0:0

Disk /dev/disks/vmhba32:0:0:0: 549.7 GB, 549763153920 bytes
64 heads, 32 sectors/track, 524295 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Device Boot    Start       End    Blocks   Id  System
/tmp # fdisk /dev/disks/vmhba32:0:0:0

The number of cylinders for this disk is set to 524295.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/disks/vmhba32:0:0:0: 549.7 GB, 549763153920 bytes
64 heads, 32 sectors/track, 524295 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Device Boot    Start       End    Blocks   Id  System

Command (m for help): n
Command action
e   extended
p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-524295, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-524295, default 524295):
Using default value 524295

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fb
Changed system type of partition 1 to fb (VMFS)

Command (m for help): p

Disk /dev/disks/vmhba32:0:0:0: 549.7 GB, 549763153920 bytes
64 heads, 32 sectors/track, 524295 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Device Boot    Start       End    Blocks   Id  System
/dev/disks/vmhba32:0:0:1             1    524295 536878064   fb  VMFS

Command (m for help): x

Expert command (m for help): b
Partition number (1-4): 1
New beginning of data (32-1073756159, default 32): 128

Expert command (m for help): p

Disk /dev/disks/vmhba32:0:0:0: 64 heads, 32 sectors, 524295 cylinders

Nr AF  Hd Sec  Cyl  Hd Sec  Cyl    Start     Size ID
1 00   1   1    0  63  32 1023        128 1073756032 fb
2 00   0   0    0   0   0    0          0          0 00
3 00   0   0    0   0   0    0          0          0 00
4 00   0   0    0   0   0    0          0          0 00

Expert command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
/tmp # vmkfstools -V
/tmp # esxcfg-vmhbadevs -m
vmhba32:0:0:1   /vmfs/devices/disks/vmhba32:0:0:1 42240509-d9375061-f3bd-0018fe7e542e
/tmp #