| Version 7 (modified by mitty, 13 years ago) (diff) |
|---|
09/14
- Ubuntu 10.04 (x86)へVanilla Kernel 2.6.35の導入
- VMゲストは、http://www.quotrader.org/vm/ubuntu1004t/ を使用
download kernel source and build
- windell57:linux s0711489$ wget https://github.com/torvalds/linux/tarball/v2.6.35 -O torvalds-linux-v2.6.35-0-g9fe6206.tar.gz --no-check-certificate
- windell57:linux s0711489$ tar xzfv torvalds-linux-v2.6.35-0-g9fe6206.tar.gz
- windell57:linux s0711489$ cd torvalds-linux-d786bf1/
- windell57:torvalds-linux-d786bf1 s0711489$ export INSTALL_PATH=/home/ugrad/07/s0711489/build/boot/
- windell57:torvalds-linux-d786bf1 s0711489$ export INSTALL_MOD_PATH=/home/ugrad/07/s0711489/build/
- windell57:torvalds-linux-d786bf1 s0711489$ make defconfig
- windell57:torvalds-linux-d786bf1 s0711489$ make menuconfig
- 記録忘れ
- 覚えている限りだと、
- Networking support からIPv6とWirelessなどの削除
- Device Drivers へdevtmpfsとSCSI low-level driversのBusLogic SCSI supportの導入
- File systems からNetwork File Systemsの削除
- Kernel hacking へCompile the kernel with debug infoの導入
- Virtualizationの削除
- windell57:torvalds-linux-d786bf1 s0711489$ make -j2
- windell57:torvalds-linux-d786bf1 s0711489$ make modules
- windell57:torvalds-linux-d786bf1 s0711489$ make install
sh /tmp/linux/torvalds-linux-d786bf1/arch/x86/boot/install.sh 2.6.35 arch/x86/boot/bzImage \ System.map "/home/ugrad/07/s0711489/build/boot/" ln: `/home/ugrad/07/s0711489/build/boot///home/ugrad/07/s0711489/build/boot//System.map-2.6.35' へのシンボリックリンク `/boot/System.map' を作成します: 許可がありません ln: `/home/ugrad/07/s0711489/build/boot///home/ugrad/07/s0711489/build/boot//vmlinuz-2.6.35' へのシンボリックリンク `/boot/vmlinuz' を作成します: 許可がありません ln: `/home/ugrad/07/s0711489/build/boot///home/ugrad/07/s0711489/build/boot//System.map-2.6.35' へのシンボリックリンク `/boot/System.map' を作成します: 許可がありません- どうやら環境変数を指定してのmake installは古いやり方らしく(?)、一部エラーとなる
- windell57:torvalds-linux-d786bf1 s0711489$ make modules_install
INSTALL arch/x86/kernel/test_nx.ko INSTALL drivers/scsi/scsi_wait_scan.ko INSTALL lib/crc16.ko INSTALL net/netfilter/xt_mark.ko DEPMOD 2.6.35
packaging kernel
- ~/build にコピーされた物をとりあえずtarballにし、ゲストOSにscpする
- windell57:build s0711489$ tar czf kernel-2.6.35.tar.gz *
- windell57:build s0711489$ scp kernel-2.6.35.tar.gz user@172.16.237.130:~
user@172.16.237.130's password: kernel-2.6.35.tar.gz 100% 8326KB 8.1MB/s 00:00
install kernel
- user@ubuntu:~$ sudo cp kernel-2.6.35.tar.gz /
- user@ubuntu:/$ sudo tar xzf kernel-2.6.35.tar.gz
- user@ubuntu:/$ sudo mkinitramfs -o /boot/initrd.img-2.6.35 2.6.35
- user@ubuntu:/$ sudo vim /etc/default/grub
# GRUB_HIDDEN_TIMEOUT=0 GRUB_HIDDEN_TIMEOUT_QUIET=false
- user@ubuntu:/$ sudo update-grub
Generating grub.cfg ... Found linux image: /boot/vmlinuz-2.6.35 Found initrd image: /boot/initrd.img-2.6.35 Found linux image: /boot/vmlinuz-2.6.35.old Found initrd image: /boot/initrd.img-2.6.35 Found linux image: /boot/vmlinuz-2.6.32-33-generic Found initrd image: /boot/initrd.img-2.6.32-33-generic Found memtest86+ image: /boot/memtest86+.bin done
結論
- 起動には成功
- NICのドライバ(PCnet32)を設定し忘れたため、ゲスト側でNICが使えずはまる(気づかずにこの日を終える)
- gdbを使ったリモートデバッグの動作確認もOK
- windell57:torvalds-linux-d786bf1 s0711489$ gdb
(gdb) file vmlinux Reading symbols from /tmp/linux/torvalds-linux-d786bf1/vmlinux...(no debugging symbols found)...done. (gdb) target remote localhost:8832 Remote debugging using localhost:8832 0xc1007c23 in native_safe_halt () at /tmp/linux/torvalds-linux-d786bf1/arch/x86/include/asm/irqflags.h:49 49 asm volatile("sti; hlt": : :"memory"); (gdb) n default_idle () at arch/x86/kernel/process.c:386 386 current_thread_info()->status |= TS_POLLING; (gdb) 392 } (gdb) cpu_idle () at arch/x86/kernel/process_32.c:101 101 while (!need_resched()) { (gdb) 115 tick_nohz_restart_sched_tick(); (gdb) 117 schedule(); (gdb) 100 tick_nohz_stop_sched_tick(1); (gdb) 101 while (!need_resched()) { (gdb) 115 tick_nohz_restart_sched_tick(); (gdb) 117 schedule(); (gdb) 100 tick_nohz_stop_sched_tick(1); (gdb) detach Ending remote debugging. (gdb) quit
- windell57:torvalds-linux-d786bf1 s0711489$ gdb
09/15
- 方針
- ゲスト上でビルドすると遅いので、ホスト上でビルドしたソースツリーをVMwareのHGFSを使ってゲスト上にマウントし、そこからsudo make installする
build kernel on Guest
- 比較対象として、同時並行でゲスト上でビルド＆インストールも行ってみた
- mitty@ubuntu-lucid:~$ sudo aptitude update
- mitty@ubuntu-lucid:~$ sudo aptitude install build-essential kernel-package libncurses-dev
Note: selecting "libncurses5-dev" instead of the virtual package "libncurses-dev" The following NEW packages will be installed: binutils{a} build-essential cvs{a} dpkg-dev{a} fakeroot{a} g++{a} g++-4.4{a} gcc{a} gcc-4.4{a} gettext{a} intltool-debian{a} kernel-package libc-dev-bin{a} libc6-dev{a} libcroco3{a} libfile-copy-recursive-perl{a} libgomp1{a} liblzma1{a} libmail-sendmail-perl{a} libncurses5-dev libstdc++6-4.4-dev{a} libsys-hostname-long-perl{a} linux-libc-dev{a} manpages-dev{a} patch{a} po-debconf{a} update-inetd{a} xz-utils{a} 0 packages upgraded, 28 newly installed, 0 to remove and 0 not upgraded. Need to get 25.7MB of archives. After unpacking 88.4MB will be used.
- mitty@ubuntu-lucid:~$ wget https://github.com/mirrors/linux/tarball/v2.6.35 -O mirrors-linux-v2.6.35-0-g9fe6206.tar.gz
- mitty@ubuntu-lucid:~/kernel$ tar xzfv mirrors-linux-v2.6.35-0-g9fe6206.tar.gz
- mitty@ubuntu-lucid:~/kernel/mirrors-linux-d786bf1$ make defconfig
- mitty@ubuntu-lucid:~/kernel/mirrors-linux-d786bf1$ make menuconfig
- 設定内容については後述
- mitty@ubuntu-lucid:~/kernel/mirrors-linux-d786bf1$ make
- mitty@ubuntu-lucid:~/kernel/mirrors-linux-d786bf1$ make modules
- mitty@ubuntu-lucid:~/kernel/mirrors-linux-d786bf1$ sudo make install
- mitty@ubuntu-lucid:~/kernel/mirrors-linux-d786bf1$ sudo make modules_install
- mitty@ubuntu-lucid:~/kernel/mirrors-linux-d786bf1$ sudo mkinitramfs -o /boot/initrd.img-2.6.35 2.6.35
- mitty@ubuntu-lucid:~/kernel/mirrors-linux-d786bf1$ sudo update-grub
- mitty@ubuntu-lucid:~$ sudo reboot
- mitty@ubuntu-lucid:~$ uname -a
Linux ubuntu-lucid 2.6.35 #3 SMP Fri Sep 16 15:01:05 JST 2011 i686 GNU/Linux
- ビルド成功
- ただ、ホスト側にはソースツリーが無いので、このままではリモートデバッグ出来ない
build kernel on Host
install packages to Guest
- s0711489@ubuntu-lucid:~$ sudo aptitude update
- s0711489@ubuntu-lucid:~$ sudo aptitude install build-essential kernel-package libncurses-dev
Note: selecting "libncurses5-dev" instead of the virtual package "libncurses-dev" The following NEW packages will be installed: binutils{a} build-essential cvs{a} dpkg-dev{a} fakeroot{a} g++{a} g++-4.4{a} gcc{a} gcc-4.4{a} gettext{a} intltool-debian{a} kernel-package libc-dev-bin{a} libc6-dev{a} libcroco3{a} libfile-copy-recursive-perl{a} libgomp1{a} liblzma1{a} libmail-sendmail-perl{a} libncurses5-dev libstdc++6-4.4-dev{a} libsys-hostname-long-perl{a} linux-libc-dev{a} manpages-dev{a} patch{a} po-debconf{a} update-inetd{a} xz-utils{a} 0 packages upgraded, 28 newly installed, 0 to remove and 0 not upgraded. Need to get 25.7MB of archives. After unpacking 88.4MB will be used.
install VMware Tools
- s0711489@ubuntu-lucid:~$ sudo mount /dev/cdrom /media/
- s0711489@ubuntu-lucid:~$ tar xzf /media/VMwareTools-8.4.6-385536.tar.gz
- s0711489@ubuntu-lucid:~/vmware-tools-distrib$ sudo ./vmware-install.pl --default
download kernel source and build
- windell57:linux-v2.6.35 s0711489$ wget https://github.com/mirrors/linux/tarball/v2.6.35 -O mirrors-linux-v2.6.35-0-g9fe6206.tar.gz --no-check-certificate
- windell57:linux-v2.6.35 s0711489$ tar xzfv mirrors-linux-v2.6.35-0-g9fe6206.tar.gz
- windell57:mirrors-linux-d786bf1 s0711489$ make defconfig
- windell57:mirrors-linux-d786bf1 s0711489$ make menuconfig
│ │ [*] Networking support ---> │ │ │ │ Networking options ---> │ │ │ │ < > The IPv6 protocol ---> │ │ │ │ [ ] Amateur Radio support ---> │ │ │ │ -*- Wireless ---> │ │ │ │ < > cfg80211 - wireless configuration API │ │ │ │ < > RF switch subsystem support ---> │ │ │ │ Device Drivers ---> │ │ │ │ Generic Driver Options ---> │ │ │ │ [*] Maintain a devtmpfs filesystem to mount at /dev │ │ │ │ [*] Automount devtmpfs at /dev, after the kernel mounted the rootfs │ │ │ │ SCSI device support ---> │ │ │ │ [*] SCSI low-level drivers ---> │ │ │ │ <*> BusLogic SCSI support │ │ │ │ [*] Network device support ---> │ │ │ │ [*] Ethernet (10 or 100Mbit) ---> │ │ │ │ <*> AMD PCnet32 PCI support │ │ │ │ File systems ---> │ │ │ │ <*> The Extended 4 (ext4) filesystem │ │ │ │ [ ] Network File Systems ---> │ │ │ │ Kernel hacking ---> │ │ │ │ [*] Compile the kernel with debug info │ │ │ │ [ ] Virtualization ---> │ │
- windell57:mirrors-linux-d786bf1 s0711489$ make -j2
- windell57:mirrors-linux-d786bf1 s0711489$ make modules
make install on Guest
- s0711489@ubuntu-lucid:~$ cd /mnt/hgfs/linux-v2.6.35/mirrors-linux-d786bf1/
- s0711489@ubuntu-lucid:/mnt/hgfs/linux-v2.6.35/mirrors-linux-d786bf1$ sudo make install
sh /mnt/hgfs/linux-v2.6.35/mirrors-linux-d786bf1/arch/x86/boot/install.sh 2.6.35 arch/x86/boot/bzImage \ System.map "/boot" - s0711489@ubuntu-lucid:/mnt/hgfs/linux-v2.6.35/mirrors-linux-d786bf1$ sudo make modules_install
INSTALL arch/x86/kernel/test_nx.ko INSTALL drivers/scsi/scsi_wait_scan.ko INSTALL fs/ext4/ext4.ko INSTALL fs/jbd2/jbd2.ko INSTALL lib/crc16.ko INSTALL net/netfilter/xt_mark.ko DEPMOD 2.6.35
- s0711489@ubuntu-lucid:/mnt/hgfs/linux-v2.6.35/mirrors-linux-d786bf1$ sudo mkinitramfs -o /boot/initrd.img-2.6.35 2.6.35
- s0711489@ubuntu-lucid:/mnt/hgfs/linux-v2.6.35/mirrors-linux-d786bf1$ sudo update-grub
Generating grub.cfg ... Found linux image: /boot/vmlinuz-2.6.35 Found initrd image: /boot/initrd.img-2.6.35 Found linux image: /boot/vmlinuz-2.6.35.old Found initrd image: /boot/initrd.img-2.6.35 Found linux image: /boot/vmlinuz-2.6.32-33-generic Found initrd image: /boot/initrd.img-2.6.32-33-generic Found memtest86+ image: /boot/memtest86+.bin done
reboot
- s0711489@ubuntu-lucid:/mnt/hgfs/linux-v2.6.35/mirrors-linux-d786bf1$ uname -a
Linux ubuntu-lucid 2.6.32-33-generic #72-Ubuntu SMP Fri Jul 29 21:08:37 UTC 2011 i686 GNU/Linux
- s0711489@ubuntu-lucid:/mnt/hgfs/linux-v2.6.35/mirrors-linux-d786bf1$ sudo reboot
- 起動しない
- 注: 後日同じ(と思われる)手順で繰り返したところ、再現せず問題なく起動した
retry with 2.6.32
- windell57:linux-v2.6.32 s0711489$ wget https://github.com/mirrors/linux/tarball/v2.6.32 -O mirrors-linux-v2.6.32-0-g22763c5.tar.gz --no-check-certificate
- windell57:mirrors-linux-459b3d5 s0711489$ make defconfig
- windell57:mirrors-linux-459b3d5 s0711489$ make menuconfig
- s0711489@ubuntu-lucid:/mnt/hgfs/linux-v2.6.32/mirrors-linux-459b3d5$ sudo make install
sh /mnt/hgfs/linux-v2.6.32/mirrors-linux-459b3d5/arch/x86/boot/install.sh 2.6.32 arch/x86/boot/bzImage \ System.map "/boot" - s0711489@ubuntu-lucid:/mnt/hgfs/linux-v2.6.32/mirrors-linux-459b3d5$ sudo make modules_install
INSTALL arch/x86/kernel/test_nx.ko INSTALL drivers/scsi/scsi_wait_scan.ko DEPMOD 2.6.32
- s0711489@ubuntu-lucid:/mnt/hgfs/linux-v2.6.32/mirrors-linux-459b3d5$ sudo mkinitramfs -o /boot/initrd.img-2.6.32 2.6.32
- s0711489@ubuntu-lucid:/mnt/hgfs/linux-v2.6.32/mirrors-linux-459b3d5$ sudo update-grub
Generating grub.cfg ... Found linux image: /boot/vmlinuz-2.6.35 Found initrd image: /boot/initrd.img-2.6.35 Found linux image: /boot/vmlinuz-2.6.35.old Found initrd image: /boot/initrd.img-2.6.35 Found linux image: /boot/vmlinuz-2.6.32-33-generic Found initrd image: /boot/initrd.img-2.6.32-33-generic Found linux image: /boot/vmlinuz-2.6.32 Found initrd image: /boot/initrd.img-2.6.32 Found memtest86+ image: /boot/memtest86+.bin done
- s0711489@ubuntu-lucid:~$ uname -a
Linux ubuntu-lucid 2.6.32-33-generic #72-Ubuntu SMP Fri Jul 29 21:08:37 UTC 2011 i686 GNU/Linux
- s0711489@ubuntu-lucid:/mnt/hgfs/linux-v2.6.32/mirrors-linux-459b3d5$ sudo reboot
- s0711489@ubuntu-lucid:~$ uname -a
Linux ubuntu-lucid 2.6.32 #1 SMP Fri Sep 16 15:49:51 JST 2011 i686 GNU/Linux
- 今度は起動成功
結論
- 2.6.35で何故失敗したかは不明(再現しなかった)
- makeにかかる時間が長すぎるので、violaを使ってビルドする方法を探る
09/17
- 高速なviolaを用いてビルドする
- Xeon X5570 * 2, 12GB memory
remake Ubuntu Guest
- 何度もkernelの導入を試してごちゃごちゃしてきてしまったので、一度再インストールしてVMのスナップショットを取っておく
- s0711489@ubuntu-lucid:~$ sudo aptitude install -R vim screen subversion
- s0711489@ubuntu-lucid:~$ sudo update-alternatives --config editor
- s0711489@ubuntu-lucid:~$ sudo visudo
- s0711489@ubuntu-lucid:~$ sudo aptitude install build-essential kernel-package libncurses-dev
- s0711489@ubuntu-lucid:~$ sudo mount /dev/cdrom /media
- s0711489@ubuntu-lucid:~$ tar xzf /media/VMwareTools-8.4.6-385536.tar.gz
- s0711489@ubuntu-lucid:~/vmware-tools-distrib$ sudo ./vmware-install.pl --default
build kernel on viola
- viola06:linux s0711489$ sha1sum -b mirrors-linux-v2.6.32-0-g22763c5.tar.gz
61a48874ea6e0887e0bd89505b80d590a63175c4 *mirrors-linux-v2.6.32-0-g22763c5.tar.gz
- viola06:mirrors-linux-459b3d5 s0711489$ make defconfig
- viola06:mirrors-linux-459b3d5 s0711489$ make menuconfig
- 前回と同じ
- viola06:mirrors-linux-459b3d5 s0711489$ make -j 16
- viola06:mirrors-linux-459b3d5 s0711489$ make modules
make install on Guest
- s0711489@ubuntu-lucid:/mnt/hgfs/mirrors-linux-459b3d5$ sudo make install
HOSTCC scripts/basic/fixdep HOSTCC scripts/basic/docproc HOSTCC scripts/basic/hash HOSTCC scripts/kconfig/conf.o scripts/kconfig/conf.c: In function ‘conf_askvalue’: scripts/kconfig/conf.c:105: warning: ignoring return value of ‘fgets’, declared with attribute warn_unused_result scripts/kconfig/conf.c: In function ‘conf_choice’: scripts/kconfig/conf.c:307: warning: ignoring return value of ‘fgets’, declared with attribute warn_unused_result HOSTCC scripts/kconfig/kxgettext.o HOSTCC scripts/kconfig/zconf.tab.o HOSTLD scripts/kconfig/conf scripts/kconfig/conf -s arch/x86/Kconfig * * Restart config... * * * RCU Subsystem * RCU Implementation > 1. Tree-based hierarchical RCU (TREE_RCU) choice[1]: 1 Enable tracing for RCU (RCU_TRACE) [N/y/?] n Tree-based hierarchical RCU fanout value (RCU_FANOUT) [32] (NEW)
- うまくいかない
- violaはwindellと違い、x86_64版のCentOSだったため、そのままビルドすると64bitカーネルになっていた
- ゲスト上でmake menuconfig & makeし直すと起動する
- うまくいかない
setup x64 version of Ubuntu
- s0711489@ubuntu-lucid64:~$ sudo aptitude install -R vim screen subversion
- s0711489@ubuntu-lucid64:~$ sudo update-alternatives --config editor
- s0711489@ubuntu-lucid64:~$ sudo visudo
- s0711489@ubuntu-lucid64:~$ sudo aptitude install build-essential kernel-package libncurses-dev
- s0711489@ubuntu-lucid64:~$ sudo mount /dev/cdrom /media
- s0711489@ubuntu-lucid64:~$ tar xzf /media/VMwareTools-8.4.6-385536.tar.gz
- s0711489@ubuntu-lucid64:~/vmware-tools-distrib$ sudo ./vmware-install.pl --default
- s0711489@ubuntu-lucid64:/mnt/hgfs/mirrors-linux-459b3d5$ sudo make install
sh /mnt/hgfs/mirrors-linux-459b3d5/arch/x86/boot/install.sh 2.6.32 arch/x86/boot/bzImage \ System.map "/boot" - s0711489@ubuntu-lucid64:/mnt/hgfs/mirrors-linux-459b3d5$ sudo make modules_install
INSTALL arch/x86/kernel/test_nx.ko INSTALL drivers/scsi/scsi_wait_scan.ko DEPMOD 2.6.32
- s0711489@ubuntu-lucid64:/mnt/hgfs/mirrors-linux-459b3d5$ sudo mkinitramfs -o /boot/initrd.img-2.6.32 2.6.32
- s0711489@ubuntu-lucid64:/mnt/hgfs/mirrors-linux-459b3d5$ sudo vim /etc/default/grub
#GRUB_HIDDEN_TIMEOUT=0 GRUB_HIDDEN_TIMEOUT_QUIET=false
- s0711489@ubuntu-lucid64:/mnt/hgfs/mirrors-linux-459b3d5$ sudo update-grub
Generating grub.cfg ... Found linux image: /boot/vmlinuz-2.6.32-33-generic Found initrd image: /boot/initrd.img-2.6.32-33-generic Found linux image: /boot/vmlinuz-2.6.32 Found initrd image: /boot/initrd.img-2.6.32 Found memtest86+ image: /boot/memtest86+.bin done
- s0711489@ubuntu-lucid64:~$ uname -a
Linux ubuntu-lucid64 2.6.32-33-generic #72-Ubuntu SMP Fri Jul 29 21:07:13 UTC 2011 x86_64 GNU/Linux
- s0711489@ubuntu-lucid64:~$ sudo reboot
- 起動しない
install Fusion-MPT SCSI device
- 64bitゲストではBusLogicが使用出来ないため、VM作成時デフォルトではLSI Logicとなるが、具体的には以下のデバイスにHDDが接続されている形となる
- s0711489@ubuntu-lucid64:~$ lspci
00:10.0 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 01)
- s0711489@ubuntu-lucid64:~$ lspci
- see http://www.linuxforums.org/forum/kernel/169338-linux-kernel-compilation-help.html#post804372
This is your SCSI controller for the harddrive. VMWare actually doesn't use SATA by default. To use this you need to enable CONFIG_FUSION_SPI
- Code:
Device Drivers ---> {*} Fusion MPT device support ---> <*> Fusion MPT ScsiHost drivers for SPI (NEW)
- Code:
- 以下のドライバを新たに導入する(ついでにBuslogicのドライバは削除しておく)
- viola06:mirrors-linux-459b3d5 s0711489$ make menuconfig
│ │ Device Drivers ---> │ │ │ │ SCSI device support ---> │ │ │ │ [ ] SCSI low-level drivers ---> │ │ │ │ [*] Fusion MPT device support ---> │ │ │ │ <*> Fusion MPT ScsiHost drivers for SPI │ │
- s0711489@ubuntu-lucid64:~$ sudo reboot
- s0711489@ubuntu-lucid64:~$ uname -a
Linux ubuntu-lucid64 2.6.32 #3 SMP Sat Sep 17 14:16:07 JST 2011 x86_64 GNU/Linux
- ビルド成功
- ビルド成功
09/25
a whole build with Vanilla Kernel 2.6.35
- x86_64はviola、i386はwindellでビルド
check signature of kernel source
- ubuntu-10.04.1-server を grsecurity のパッチを当てたカーネルで再構築する | guro_chanの日記
- The Linux Kernel Archives PGP Signature
- mitty@ubuntu64:~$ gpg --keyserver wwwkeys.pgp.net --recv-keys 0x517D0F0E
gpg: requesting key 517D0F0E from hkp server wwwkeys.pgp.net gpg: /home/mitty/.gnupg/trustdb.gpg: trustdb created gpg: key 517D0F0E: public key "Linux Kernel Archives Verification Key <ftpadmin@kernel.org>" imported gpg: no ultimately trusted keys found gpg: Total number processed: 1 gpg: imported: 1
- mitty@ubuntu64:~/vendor/KernelHack$ wget http://ftp.jaist.ac.jp/pub/Linux/kernel.org/linux/kernel/v2.6/longterm/v2.6.35/linux-2.6.35.14.tar.gz
- mitty@ubuntu64:~/vendor/KernelHack$ wget http://ftp.jaist.ac.jp/pub/Linux/kernel.org/linux/kernel/v2.6/longterm/v2.6.35/linux-2.6.35.14.tar.gz.sign
- mitty@ubuntu64:~/vendor/KernelHack$ gpg --verify linux-2.6.35.14.tar.gz.sign linux-2.6.35.14.tar.gz
gpg: Signature made Tue 02 Aug 2011 06:36:48 AM JST using DSA key ID 517D0F0E gpg: Good signature from "Linux Kernel Archives Verification Key <ftpadmin@kernel.org>" gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: C75D C40A 11D7 AF88 9981 ED5B C86B A06A 517D 0F0E
extract source tarball
- viola05:KernelHack s0711489$ tar xzf linux-2.6.35.14.tar.gz
- viola05:KernelHack s0711489$ mv linux-2.6.35.14/ x86_64
- viola05:KernelHack s0711489$ tar xzf linux-2.6.35.14.tar.gz
- viola05:KernelHack s0711489$ mv linux-2.6.35.14/ i386
- viola05:KernelHack s0711489$ mkdir linux-2.6.35.14
- viola05:KernelHack s0711489$ mv x86_64 i386 linux-2.6.35.14/
make defconfig
- windell57:i386 s0711489$ make defconfig
HOSTCC scripts/basic/fixdep HOSTCC scripts/basic/docproc HOSTCC scripts/basic/hash HOSTCC scripts/kconfig/conf.o HOSTCC scripts/kconfig/kxgettext.o SHIPPED scripts/kconfig/zconf.tab.c SHIPPED scripts/kconfig/lex.zconf.c SHIPPED scripts/kconfig/zconf.hash.c HOSTCC scripts/kconfig/zconf.tab.o HOSTLD scripts/kconfig/conf *** Default configuration is based on 'i386_defconfig' # # configuration written to .config #
- viola05:x86_64 s0711489$ make defconfig
HOSTCC scripts/basic/fixdep HOSTCC scripts/basic/docproc HOSTCC scripts/basic/hash HOSTCC scripts/kconfig/conf.o HOSTCC scripts/kconfig/kxgettext.o SHIPPED scripts/kconfig/zconf.tab.c SHIPPED scripts/kconfig/lex.zconf.c SHIPPED scripts/kconfig/zconf.hash.c HOSTCC scripts/kconfig/zconf.tab.o HOSTLD scripts/kconfig/conf *** Default configuration is based on 'x86_64_defconfig' # # configuration written to .config #
make menuconfig
x86_64
- viola05:x86_64 s0711489$ make menuconfig
HOSTCC scripts/kconfig/lxdialog/checklist.o HOSTCC scripts/kconfig/lxdialog/inputbox.o HOSTCC scripts/kconfig/lxdialog/menubox.o HOSTCC scripts/kconfig/lxdialog/textbox.o HOSTCC scripts/kconfig/lxdialog/util.o HOSTCC scripts/kconfig/lxdialog/yesno.o HOSTCC scripts/kconfig/mconf.o HOSTLD scripts/kconfig/mconf scripts/kconfig/mconf arch/x86/Kconfig
.config - Linux Kernel v2.6.35.14 Configuration │ │ [*] Networking support ---> │ │ │ │ Networking options ---> │ │ │ │ < > The IPv6 protocol ---> │ │ │ │ [ ] Amateur Radio support ---> │ │ │ │ -*- Wireless ---> │ │ │ │ < > cfg80211 - wireless configuration API │ │ │ │ < > RF switch subsystem support ---> │ │ │ │ Device Drivers ---> │ │ │ │ Generic Driver Options ---> │ │ │ │ [*] Maintain a devtmpfs filesystem to mount at /dev │ │ │ │ [*] Automount devtmpfs at /dev, after the kernel mounted the rootfs │ │ │ │ [ ] Multiple devices driver support (RAID and LVM) ---> │ │ │ │ [*] Fusion MPT device support ---> │ │ │ │ <*> Fusion MPT ScsiHost drivers for SPI │ │ │ │ [ ] Macintosh device drivers ---> │ │ │ │ [*] Network device support ---> │ │ │ │ [*] Ethernet (10 or 100Mbit) ---> │ │ │ │ <*> AMD PCnet32 PCI support │ │ │ │ [ ] Ethernet (1000 Mbit) ---> │ │ │ │ [ ] Ethernet (10000 Mbit) ---> │ │ │ │ < > Token Ring driver support ---> │ │ │ │ [ ] Wireless LAN ---> │ │ │ │ [ ] PCMCIA network device support ---> │ │ │ │ < > FDDI driver support │ │ │ │ < > Sound card support ---> │ │ │ │ [ ] HID Devices ---> │ │ │ │ [ ] USB support ---> │ │ │ │ File systems ---> │ │ │ │ <*> The Extended 4 (ext4) filesystem │ │ │ │ [ ] Network File Systems ---> │ │ │ │ Kernel hacking ---> │ │ │ │ [*] Compile the kernel with debug info │ │ │ │ Security options ---> │ │ │ │ [ ] NSA SELinux Support │ │ │ │ [ ] Virtualization ---> │ │
i386
- windell57:i386 s0711489$ make menuconfig
HOSTCC scripts/kconfig/lxdialog/checklist.o HOSTCC scripts/kconfig/lxdialog/inputbox.o HOSTCC scripts/kconfig/lxdialog/menubox.o HOSTCC scripts/kconfig/lxdialog/textbox.o HOSTCC scripts/kconfig/lxdialog/util.o HOSTCC scripts/kconfig/lxdialog/yesno.o HOSTCC scripts/kconfig/mconf.o HOSTLD scripts/kconfig/mconf scripts/kconfig/mconf arch/x86/Kconfig
.config - Linux Kernel v2.6.35.14 Configuration │ │ [*] Networking support ---> │ │ │ │ Networking options ---> │ │ │ │ < > The IPv6 protocol ---> │ │ │ │ [ ] Amateur Radio support ---> │ │ │ │ -*- Wireless ---> │ │ │ │ < > cfg80211 - wireless configuration API │ │ │ │ < > RF switch subsystem support ---> │ │ │ │ Device Drivers ---> │ │ │ │ Generic Driver Options ---> │ │ │ │ [*] Maintain a devtmpfs filesystem to mount at /dev │ │ │ │ [*] Automount devtmpfs at /dev, after the kernel mounted the rootfs │ │ │ │ SCSI device support ---> │ │ │ │ [*] SCSI low-level drivers ---> │ │ │ │ <*> BusLogic SCSI support │ │ │ │ [ ] Multiple devices driver support (RAID and LVM) ---> │ │ │ │ [ ] Macintosh device drivers ---> │ │ │ │ [*] Network device support ---> │ │ │ │ [*] Ethernet (10 or 100Mbit) ---> │ │ │ │ <*> AMD PCnet32 PCI support │ │ │ │ [ ] Ethernet (1000 Mbit) ---> │ │ │ │ [ ] Ethernet (10000 Mbit) ---> │ │ │ │ < > Token Ring driver support ---> │ │ │ │ [ ] Wireless LAN ---> │ │ │ │ [ ] PCMCIA network device support ---> │ │ │ │ < > FDDI driver support │ │ │ │ < > Sound card support ---> │ │ │ │ [ ] HID Devices ---> │ │ │ │ [ ] USB support ---> │ │ │ │ File systems ---> │ │ │ │ <*> The Extended 4 (ext4) filesystem │ │ │ │ [ ] Network File Systems ---> │ │ │ │ Kernel hacking ---> │ │ │ │ [*] Compile the kernel with debug info │ │ │ │ Security options ---> │ │ │ │ [ ] NSA SELinux Support │ │ │ │ [ ] Virtualization ---> │ │
make and make install to guest
i386
make
- windell57:i386 s0711489$ make -j2
- windell57:i386 s0711489$ make modules
change grub settings to display OS entries
- s0711489@ubuntu-lucid:~$ sudo vim /etc/default/grub
-
/etc/default/grub
old new 2 2 # /boot/grub/grub.cfg. 3 3 4 4 GRUB_DEFAULT=0 5 GRUB_HIDDEN_TIMEOUT=06 GRUB_HIDDEN_TIMEOUT_QUIET= true5 #GRUB_HIDDEN_TIMEOUT=0 6 GRUB_HIDDEN_TIMEOUT_QUIET=false 7 7 GRUB_TIMEOUT=10 8 8 GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian` 9 9 GRUB_CMDLINE_LINUX_DEFAULT="splash quiet"
-
make install
- s0711489@ubuntu-lucid:/mnt/hgfs/linux-2.6.35.14$ sudo make install
sh /mnt/hgfs/linux-2.6.35.14/arch/x86/boot/install.sh 2.6.35.14 arch/x86/boot/bzImage \ System.map "/boot" - s0711489@ubuntu-lucid:/mnt/hgfs/linux-2.6.35.14$ sudo make modules_install
INSTALL arch/x86/kernel/test_nx.ko INSTALL drivers/scsi/scsi_wait_scan.ko INSTALL net/netfilter/xt_mark.ko DEPMOD 2.6.35.14
- s0711489@ubuntu-lucid:~$ sudo mkinitramfs -o /boot/initrd.img-2.6.35.14 2.6.35.14
- s0711489@ubuntu-lucid:~$ sudo update-grub
Generating grub.cfg ... Found linux image: /boot/vmlinuz-2.6.35.14 Found initrd image: /boot/initrd.img-2.6.35.14 Found linux image: /boot/vmlinuz-2.6.32-33-generic Found initrd image: /boot/initrd.img-2.6.32-33-generic Found memtest86+ image: /boot/memtest86+.bin done
reboot
- s0711489@ubuntu-lucid:~$ sudo reboot
- s0711489@ubuntu-lucid:~$ uname -a
Linux ubuntu-lucid 2.6.35.14 #1 SMP Sun Sep 25 20:00:16 JST 2011 i686 GNU/Linux
x86_64
make
- viola05:x86_64 s0711489$ make -j16
- viola05:x86_64 s0711489$ make modules
change grub settings to display OS entries
- s0711489@ubuntu-lucid64:~$ sudo vim /etc/default/grub
-
/etc/default/grub
old new 2 2 # /boot/grub/grub.cfg. 3 3 4 4 GRUB_DEFAULT=0 5 GRUB_HIDDEN_TIMEOUT=06 GRUB_HIDDEN_TIMEOUT_QUIET= true5 #GRUB_HIDDEN_TIMEOUT=0 6 GRUB_HIDDEN_TIMEOUT_QUIET=false 7 7 GRUB_TIMEOUT=10 8 8 GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian` 9 9 GRUB_CMDLINE_LINUX_DEFAULT="splash quiet"
-
make install
- s0711489@ubuntu-lucid64:/mnt/hgfs/linux-2.6.35.14$ sudo make install
sh /mnt/hgfs/linux-2.6.35.14/arch/x86/boot/install.sh 2.6.35.14 arch/x86/boot/bzImage \ System.map "/boot" - s0711489@ubuntu-lucid64:/mnt/hgfs/linux-2.6.35.14$ sudo make modules_install
INSTALL arch/x86/kernel/test_nx.ko INSTALL drivers/scsi/scsi_wait_scan.ko INSTALL net/netfilter/xt_mark.ko DEPMOD 2.6.35.14
- s0711489@ubuntu-lucid64:~$ sudo mkinitramfs -o /boot/initrd.img-2.6.35.14 2.6.35.14
- s0711489@ubuntu-lucid64:~$ sudo update-grub
Generating grub.cfg ... Found linux image: /boot/vmlinuz-2.6.35.14 Found initrd image: /boot/initrd.img-2.6.35.14 Found linux image: /boot/vmlinuz-2.6.32-33-generic Found initrd image: /boot/initrd.img-2.6.32-33-generic Found memtest86+ image: /boot/memtest86+.bin done
reboot
- s0711489@ubuntu-lucid64:~$ sudo reboot
- windell57:~ s0711489$ ssh 172.16.237.132
ssh: connect to host 172.16.237.132 port 22: No route to host
add e1000 driver
- 64bitゲストだとPCnet32の代わりにIntel Pro/1000が用意される
- s0711489@ubuntu-lucid64:~$ lspci
02:00.0 Ethernet controller: Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) (rev 01)
- viola05:x86_64 s0711489$ make menuconfig
.config - Linux Kernel v2.6.35.14 Configuration │ │ Device Drivers ---> │ │ │ │ [*] Network device support ---> │ │ │ │ [*] Ethernet (1000 Mbit) ---> │ │ │ │ <*> Intel(R) PRO/1000 Gigabit Ethernet support │ │
- viola05:x86_64 s0711489$ make -j16
- viola05:x86_64 s0711489$ make modules
- s0711489@ubuntu-lucid64:/mnt/hgfs/linux-2.6.35.14$ sudo make install
sh /mnt/hgfs/linux-2.6.35.14/arch/x86/boot/install.sh 2.6.35.14 arch/x86/boot/bzImage \ System.map "/boot" - s0711489@ubuntu-lucid64:/mnt/hgfs/linux-2.6.35.14$ sudo make modules_install
INSTALL arch/x86/kernel/test_nx.ko INSTALL drivers/scsi/scsi_wait_scan.ko INSTALL net/netfilter/xt_mark.ko DEPMOD 2.6.35.14
- s0711489@ubuntu-lucid64:~$ sudo mkinitramfs -o /boot/initrd.img-2.6.35.14 2.6.35.14
- s0711489@ubuntu-lucid64:~$ sudo update-grub
Generating grub.cfg ... Found linux image: /boot/vmlinuz-2.6.35.14 Found initrd image: /boot/initrd.img-2.6.35.14 Found linux image: /boot/vmlinuz-2.6.32-33-generic Found initrd image: /boot/initrd.img-2.6.32-33-generic Found memtest86+ image: /boot/memtest86+.bin done
- s0711489@ubuntu-lucid64:~$ sudo reboot
- s0711489@ubuntu-lucid64:~$ uname -a
Linux ubuntu-lucid64 2.6.35.14 #2 SMP Sun Sep 25 21:20:21 JST 2011 x86_64 GNU/Linux
remove PCnet32 driver
- viola05:x86_64 s0711489$ make menuconfig
.config - Linux Kernel v2.6.35.14 Configuration │ │ Device Drivers ---> │ │ │ │ [*] Network device support ---> │ │ │ │ [*] Ethernet (10 or 100Mbit) ---> │ │ │ │ < > AMD PCnet32 PCI support │ │
- viola05:x86_64 s0711489$ make -j16
- viola05:x86_64 s0711489$ make modules
- (略)
- s0711489@ubuntu-lucid64:~$ sudo reboot
- s0711489@ubuntu-lucid64:~$ uname -a
Linux ubuntu-lucid64 2.6.35.14 #3 SMP Sun Sep 25 21:32:17 JST 2011 x86_64 GNU/Linux
set remote debug option to VMware config
i386
- windell57:~ s0711489$ vim .vmware/ubuntu-lucid/ubuntu-lucid.vmx
-
ubuntu-lucid.vmx
old new 73 73 sharedFolder0.hostPath = "/home/ugrad/07/s0711489/coursework/KernelHack/linux-2.6.35.14/i386" 74 74 sharedFolder0.guestName = "linux-2.6.35.14" 75 75 sharedFolder0.expiration = "never" 76 debugStub.listen.guest32 = "TRUE" 77 debugStub.listen.guest32.remote = "TRUE"
-
- windell57:~ s0711489$ netstat -na | grep 8832
tcp 0 0 0.0.0.0:8832 0.0.0.0:* LISTEN
- debugStub.listen.guest32.remoteが無い場合、localhostにのみbindする
- windell57:~ s0711489$ netstat -na | grep 8832
tcp 0 0 127.0.0.1:8832 0.0.0.0:* LISTEN
- windell57:~ s0711489$ netstat -na | grep 8832
x86_64
- windell57:~ s0711489$ vim .vmware/ubuntu-lucid64/ubuntu-lucid64.vmx
-
ubuntu-lucid64.vmx
old new 75 75 sharedFolder0.hostPath = "/home/ugrad/07/s0711489/coursework/KernelHack/linux-2.6.35.14/x86_64" 76 76 sharedFolder0.guestName = "linux-2.6.35.14" 77 77 sharedFolder0.expiration = "never" 78 debugStub.listen.guest64 = "TRUE" 79 debugStub.listen.guest64.remote = "TRUE"
-
- windell57:~ s0711489$ netstat -na | grep 8864
tcp 0 0 0.0.0.0:8864 0.0.0.0:* LISTEN
09/28
- カーネルのビルドごとにゲスト側でインストールコマンドを叩くのが面倒なのでシェルスクリプトにした
- install.sh
#! /bin/sh -ex VERSION=`ls -d /mnt/hgfs/linux-* | sed s#.*linux-##` echo Install Linux Kernel version ${VERSION} cd /mnt/hgfs/linux-${VERSION}/ make install make modules_install mkinitramfs -o /boot/initrd.img-${VERSION} ${VERSION} update-grub echo Install Kernel Headers to /lib/modules/${VERSION}/build mv /lib/modules/${VERSION}/build /lib/modules/${VERSION}/build-`date '+%Y%m%d'` cd /mnt/hgfs/ find linux-${VERSION} -type f | egrep -v '.o$' | egrep -v 'vmlinu' | tar c --files-from - | tar x -C /lib/modules/${VERSION}/ mv /lib/modules/${VERSION}/linux-${VERSION} /lib/modules/${VERSION}/build uname -a
- s0711489@ubuntu-lucid64:~$ time sudo /mnt/hgfs/tools/install.sh
+ sed s#.*linux-## + ls -d /mnt/hgfs/linux-2.6.35.14 + VERSION=2.6.35.14 + echo Install Linux Kernel version 2.6.35.14 Install Linux Kernel version 2.6.35.14 + cd /mnt/hgfs/linux-2.6.35.14/ + make install sh /mnt/hgfs/linux-2.6.35.14/arch/x86/boot/install.sh 2.6.35.14 arch/x86/boot/bzImage \ System.map "/boot" + make modules_install INSTALL arch/x86/kernel/test_nx.ko INSTALL drivers/scsi/scsi_wait_scan.ko INSTALL net/netfilter/xt_mark.ko DEPMOD 2.6.35.14 + mkinitramfs -o /boot/initrd.img-2.6.35.14 2.6.35.14 + update-grub Generating grub.cfg ... Found linux image: /boot/vmlinuz-2.6.35.14 Found initrd image: /boot/initrd.img-2.6.35.14 Found linux image: /boot/vmlinuz-2.6.32-33-generic Found initrd image: /boot/initrd.img-2.6.32-33-generic Found memtest86+ image: /boot/memtest86+.bin done + echo Install Kernel Headers to /lib/modules/2.6.35.14/build Install Kernel Headers to /lib/modules/2.6.35.14/build + date +%Y%m%d + mv /lib/modules/2.6.35.14/build /lib/modules/2.6.35.14/build-20110928 + cd /mnt/hgfs/ + tar c --files-from - + tar x -C /lib/modules/2.6.35.14/ + egrep -v vmlinu + egrep -v .o$ + find linux-2.6.35.14 -type f + mv /lib/modules/2.6.35.14/linux-2.6.35.14 /lib/modules/2.6.35.14/build + uname -a Linux ubuntu-lucid64 2.6.32-33-generic #72-Ubuntu SMP Fri Jul 29 21:07:13 UTC 2011 x86_64 GNU/Linux real 1m32.138s user 0m13.790s sys 1m26.160s
09/30
install VMware tools to custom kernel
- install.sh
#! /bin/sh -ex VERSION=`cat /mnt/hgfs/linux-*/include/config/kernel.release` echo Install Linux Kernel version ${VERSION} cd /mnt/hgfs/linux-*/ make install make modules_install mkinitramfs -o /boot/initrd.img-${VERSION} ${VERSION} update-grub echo Install Kernel Headers to /lib/modules/${VERSION}/build mv /lib/modules/${VERSION}/build /lib/modules/${VERSION}/build-`date '+%Y%m%d'` cd /mnt/hgfs/ find linux-* -type f | egrep -v '.o$' | egrep -v 'vmlinu' | tar c --files-from - | tar x -C /lib/modules/${VERSION}/ mv /lib/modules/${VERSION}/linux-* /lib/modules/${VERSION}/build uname -a
- s0711489@ubuntu-lucid64:~$ sudo /mnt/hgfs/tools/install.sh
+ cat /mnt/hgfs/linux-2.6.35.14/include/config/kernel.release + VERSION=2.6.35.14 + echo Install Linux Kernel version 2.6.35.14 Install Linux Kernel version 2.6.35.14 + cd /mnt/hgfs/linux-2.6.35.14/ + make install sh /mnt/hgfs/linux-2.6.35.14/arch/x86/boot/install.sh 2.6.35.14 arch/x86/boot/bzImage \ System.map "/boot" + make modules_install INSTALL arch/x86/kernel/test_nx.ko INSTALL drivers/scsi/scsi_wait_scan.ko INSTALL net/netfilter/xt_mark.ko DEPMOD 2.6.35.14 + mkinitramfs -o /boot/initrd.img-2.6.35.14 2.6.35.14 + update-grub Generating grub.cfg ... Found linux image: /boot/vmlinuz-2.6.35.14 Found initrd image: /boot/initrd.img-2.6.35.14 Found linux image: /boot/vmlinuz-2.6.32-33-generic Found initrd image: /boot/initrd.img-2.6.32-33-generic Found memtest86+ image: /boot/memtest86+.bin done + echo Install Kernel Headers to /lib/modules/2.6.35.14/build Install Kernel Headers to /lib/modules/2.6.35.14/build + date +%Y%m%d + mv /lib/modules/2.6.35.14/build /lib/modules/2.6.35.14/build-20110930 + cd /mnt/hgfs/ + egrep -v .o$ + egrep -v vmlinu + tar c --files-from - + tar x -C /lib/modules/2.6.35.14/ + find linux-2.6.35.14 -type f + mv /lib/modules/2.6.35.14/linux-2.6.35.14 /lib/modules/2.6.35.14/build + uname -a Linux ubuntu-lucid64 2.6.32-33-generic #72-Ubuntu SMP Fri Jul 29 21:07:13 UTC 2011 x86_64 GNU/Linux - s0711489@ubuntu-lucid64:~$ sudo reboot
- s0711489@ubuntu-lucid64:~$ uname -a
- s0711489@ubuntu-lucid64:~$ sudo vmware-config-tools.pl --default
step execution with GDB
- s0711489@ubuntu-lucid64:~/coursework/KernelHack/02$ cat > getpid.c
#include <sys/types.h> #include <syscall.h> #include <stdio.h> int main (void) { printf ("getpid() -> %d\n", getpid()); }
- s0711489@ubuntu-lucid64:~/coursework/KernelHack/02$ cc getpid.c -o getpid
- s0711489@ubuntu-lucid64:~/coursework/KernelHack/02$ strace -f -o getpid.log ./getpid
getpid() -> 2007
- windell57:x86_64 s0711489$ gdb
(gdb) file vmlinux Reading symbols from /home/ugrad/07/s0711489/coursework/KernelHack/linux-2.6.35.14/x86_64/vmlinux...(no debugging symbols found)...done. (gdb) target remote localhost:8864
(gdb) help step Step program until it reaches a different source line. Argument N means do this N times (or till program stops for another reason). (gdb) help next Step program, proceeding through subroutine calls. Like the "step" command as long as subroutine calls do not happen; when they do, the call is treated as one instruction. Argument N means do this N times (or till program stops for another reason).
- 初めのうちはnextを使っていたため、サブルーチンコールがスキップされてしまい、うまく実行コードを見ることが出来なかった。
- windell57:02 s0711489$ cat > gdb.step.sh
echo 'file vmlinux target remote localhost:8864' yes s
- s0711489@ubuntu-lucid64:~/coursework/KernelHack/02$ while true; do ./getpid ;done
getpid() -> 1816 getpid() -> 1817 (snip) getpid() -> 2846 getpid() -> 2847 getpid() -> 2848 Killed by signal 15.
- windell57:x86_64 s0711489$ sh ../../02/gdb.step.sh | gdb > gdb.txt
- stepコマンドだけでは、実行されるコードが膨大すぎて、目当てのgetpidシステムコールまで到達出来なかった
Attachments (4)
- WS001485.png (82.0 KB) - added by mitty 13 years ago.
- WS001486.png (60.0 KB) - added by mitty 13 years ago.
- WS001488.png (80.6 KB) - added by mitty 13 years ago.
- WS001489.png (61.5 KB) - added by mitty 13 years ago.
Download all attachments as: .zip
