2010/05/03

vmci error on VMware Tools install on Ubuntu 10.04 (Linux 2.6.32) / ESXi 4

Reference: VMware Communities: Can't install latest VMware Server 2.0.2 on 2.6.32.8

Problem


升級 Ubuntu 10.04 LTS (內核 2.6.32) 後嘗試重新編譯 VMware Tools 卻發生如下的問題:
When OS kernel upgrade into Linux 2.6.32 (like Ubuntu 10.04 LTS), compiling VMware Tools will occur some problems like the below:

CC [M] /tmp/vmware-config1/vmci-only/vmciQueuePair.o
In file included from /tmp/vmware-config1/vmci-only/vmci_queue_pair.h:36,
from /tmp/vmware-config1/vmci-only/vmciQueuePair.c:36:
/tmp/vmware-config1/vmci-only/vm_atomic.h:996:7: warning: "_MSC_VER" is not defined
/tmp/vmware-config1/vmci-only/vm_atomic.h:1081:7: warning: "_MSC_VER" is not defined
In file included from /tmp/vmware-config1/vmci-only/vmci_queue_pair.h:36,
from /tmp/vmware-config1/vmci-only/vmciQueuePair.c:36:
/tmp/vmware-config1/vmci-only/vm_atomic.h:996:7: warning: "_MSC_VER" is not defined
/tmp/vmware-config1/vmci-only/vm_atomic.h:1081:7: warning: "_MSC_VER" is not defined
......

vmci 模組無法編譯成功,也無法載入至系統,更新失敗。
And vmci module will not be load into system.

Solution


1. cd /usr/lib/vmware-tools/modules/source
2. tar xvf vmci.tar (Extract it)
3. mv vmci.tar vmci.tar.bak (Backup)
4. cd vmci-only
5. vi pgtbl.h (Patch pgtbl.h)
--- vmci-only.old/include/pgtbl.h    2009-10-21 03:43:27.000000000 +0200
+++ vmci-only/include/pgtbl.h    2010-02-12 19:00:50.277528449 +0100
@@ -24,6 +24,7 @@
 #include "compat_pgtable.h"
 #include "compat_spinlock.h"
 #include "compat_page.h"
+#include "compat_sched.h"
 
 #if LINUX_VERSION_CODE < KERNEL_VERSION(2, 3, 11)
 # define compat_active_mm  mm

把 #include "compat_sched.h" 加在 #include "compat_page.h" 後面一行。存檔離開。
Add "#include "compat_sched.h"" after "#include "compat_page.h"" and save. Quit.

6. vi Makefile (Patch Makefile)
--- vmci-only.old/Makefile    2009-10-21 03:43:27.000000000 +0200
+++ vmci-only/Makefile    2010-02-12 17:59:21.470529639 +0100
@@ -113,7 +113,7 @@
 
 vm_check_build = $(shell if $(CC) $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS) \
     $(CPPFLAGS) $(CFLAGS) $(CFLAGS_KERNEL) \
-    $(EXTRA_CFLAGS) -Iinclude2/asm/mach-default \
+    $(EXTRA_CFLAGS) -I$(HEADER_DIR) -Iinclude2/asm/mach-default \
     -DKBUILD_BASENAME=\"$(DRIVER)\" \
     -Werror -S -o /dev/null -xc $(1) \
     > /dev/null 2>&1; then echo "$(2)"; else echo "$(3)"; fi)

找到這行
Replace this line:
$(EXTRA_CFLAGS) -Iinclude2/asm/mach-default \
取代成
With:
$(EXTRA_CFLAGS) -I$(HEADER_DIR) -Iinclude2/asm/mach-default \
一樣存檔退出。
Then save and quit.

7. cd .. (Quit this folder)
8. tar cvf vmci.tar vmci-only/ (Repack the source)
9. rm vmci-only -r (Delete the entire folder)
10. /usr/bin/vmware-config-tools.pl (Re-run config tool to recompile modules)

重新執行之後應該就可以順利編譯成功並載入至系統,檢查看看是否載入:
Alter all, check if these modules are loaded perfectly:
$ lsmod|grep "^v"
vsock                  38646  0
vmmemctl                8768  0
vmxnet                 15992  0
vga16fb                12757  1
vgastate                9857  1 vga16fb
vmci                   29828  1 vsock    <-- Here, good
vmw_pvscsi             15744  0
vmxnet3                30709  0