Solaris之家: edward shu's blog

王石，马云等一行人组成的临终关怀团，在看望了艾滋病患者，癌症晚期患者和敬老院之后，顺便看望了正在集训备战世界杯预选赛并且被分在死亡之组的的中国国家足球队。

晚上回家车上打开收音机,  正好是一个台的有奖问答. 规则是问题提出来之后,  然后听众竞播,  第一个进来的听众如果答对的话, 奖励100元,  答错的话机会留给下一个人,  奖金变为200元,  依此类推, 奖金逐渐增高.  主持人的第一道题是" 人身上最小的肌肉在哪".  过了一会,  有听众拨进来了,  答曰, 手上.  主持人一听,  觉得这个答案太不靠边了, 就开始提示了,  "这块肌肉是藏在某个地方, 你听我说话的时候就要用到, 是五官的一部分, 并且长在两侧".  这个提示已经很明显了. 接着第二观众来电, 答曰, "脸上". 当场喷饭.  主持人已经着急了说, "我说是长在两测, 千万别给我说眼睛, 眼睛长在两侧的是螃蟹,  脸上是能看见的，我告诉你这块肌肉的名字叫'镫骨肌',  声音过大的时候就会自动升缩，具有保护作用，　我不能再提示了".   第三个听众打进来了，　答曰,   "鼻子".  这回我已经是欲哭无泪了．　心里暗骂，　这些人太弱智了．　仔细一想，　这些人不该是故意答错，然后奖金就逐渐垒高，　最后才照单全收，　实在是高啊．　这个世界，到底是谁愚弄了谁?

Our internal Xorg SUN  is a patched soruce tree from open source community. So it is better to debug it with GNU tool, ie., gdb. As the usual, we should compile the source files with debug information first. Some kind of trick is used here. Let's give a example as the following, we like to know what is going on with the Intel video driver. The driver may have some dependency on other modules, so we build the full Xorg tree first. As a result, we get a compiled Xorg source tree without debug information. Under the directory "open-src/driver/xf86-video-intel", "make clean & make debug" command will give you a Intel driver with debug information.  Let's have a look why you can do this. There is a "Makefile" "open-src/driver/xf86-video-intel which depends on "../Makefile.inc", and "../Makefile.inc" itself depends on "../../common/Makefile". A debug target is defined in "../../common/Makefile".

  After install the new object file into target system, we can start our adventure now. If you do not tell it, GDB does not know where the source files are located. I prefer to write the source path in ".gdbinit" just as the B-shell search in path "bashrc". 

  -bash /usr/X11/bin/Xorg

  (gdb) file /usr/X11/bin/Xorg

  (gdb) file /usr/X11/bin/Xorg

Load new symbol table from "/usr/X11/bin/Xorg"? (y or n)
Reading symbols from /usr/X11/bin/Xorg...done.

(gdb) break i830_dri.c:I830DRIScreenInit

Make breakpoint pending on future shared library load? (y or [n]) y

Breakpoint 1 (i830_dri.c:I830DRIScreenInit) pending.

(gdb)

Fourtunately, "defered breakpoints" is supported by gdb, so we can set the breakpoint on the share library which haven't been loaded yet.
 

 

时常用Solaris办公，因为安装的是中文化的solaris, 里面经常会出来一些信息是中文的。比如一个管道操作失败， 系统就告诉你"打破的管道"。 初一看， "打破"这个词很吓人，就好像有人潜伏在solaris专干破坏工作一样。 仔细一琢磨，这不就是"broken pipe"的直译吗，让人苦笑不得。有没有更好的词来翻译它呢？"残破的管道", "毁坏的管道"...，想到的就是这么些了。

Currently, the video card is left randomly after it resume from the power suspend.
Neither BIOS nor video card driver restore all the device context.
Here video card driver refers to both DDX driver and DRM driver.

The context of the video cards is composed from three parts:
   1. 2D context including register status and device memory content
      Xserver charge these.
   2. 3D context including register status and device memory content
      DRI and DRM driver charge these.
   3. PCI config space.
      In Solaris, it is the responsibility of Nexus driver to save and restore all the PCI
     config space.

    Xserver and DRM driver may save and restore part of the device's register. It is highly
 possible that we will create a boundary and hand shake for Xserver and DRM driver.

    How linux does?

    In open source world, some guys use some hack method to the graphics suspend and resume.

    1). Save the configuration space of the device manually by some tools.
    2). Suspend the device, then bios will leave the device in random state
    3). Post the device and restore the configuration of the device manually by some tools.
     

   By this way, linux sometimes can work with or without Xorg. But it is very unstable.

 
 

 

In order to get more information, I used two workstation for analysis. One is the Ultra 20, the other is

IBM ThinkCenter A50P. The Solaris trial version of X86 PM can work well on Ultral 20, ie, the system can do suspend and resume properly.

Let's take a look on How ultra 20 works.
1)Setup the kmdb through serial line
2)active kmdb and set the breakpoint at AcpiEnterSleepState
3)Run command "uadmin 3 20"

the system stop at following stack:

acpica`AcpiEnterSleepState(ffffff00024a4b78)

acpippm`appm_ioctl+0x8f(de00000000, 4101, ffffff00024a4b78, 80200002,fffffffec2f0de18, 0)

cdev_ioctl+0x48(de00000000, 4101, ffffff00024a4b78, 80200002, fffffffec2f0de18, 0)

ldi_ioctl+0xac(fffffffec41e5f60, 4101, ffffff00024a4b78, 80000002, fffffffec2f0de18, 0)

ppm`ppm_manage_sx+0x88(ffffff00024a4b78, 1)

ppm`ppm_ctlops+0x121(fffffffec27eb458, fffffffec27fec78, 15, ffffff00024a4c88, ffffff00024a4cc4)

pm_ctlops+0x70(fffffffec27eb458, fffffffec27fec78, 15, ffffff00024a4c88, ffffff00024a4cc4)

cpr`i_cpr_power_down+0x2ff(3)

cpr`cpr_main+0x1aa(3)

cpr`cpr+0xdb(14)

stubs_common_code+0x51()

kadmin+0x179(3, 14, 0, fffffffed99f2d60)

uadmin+0xb1(3, 14, 0) 

4) However, when I set the defering breakpoint, ie, "::bp cpr`cpr_main" or "::bp AcpiEnterSleepState" The system can stop.
On the other hand, when I did the same thing on the IBM A50P,  It can only stop at "::bp cpr`cpr_main".

 

As we have known, Solaris does not export MTRR interfaces to user level applications. So PAT is used for write combing support for Xorg. Let's verified it by mdb.

1). Start solaris and Xorg.

2). Make sure solaris dumps all the pages beside kernel pages. dumpadm -c all

3). Reboot the solaris and get the dump core files reboot -d

4). mdb -k 0. Suppose your core files are "unix.0" and "vmcore.0".

5). find the pid of Xorg ::ps ! grep Xorg

     Here I got "100777".

6). get the HAT table of the Xorg 0t100777 ::pid2proc |::print proc_t p_as->a_hat

    Here I got "0xceb7d140".

7). dump the all the page table entries 0xceb7d140::htables | ::print struct htable ht_pfn |::ptable

8). From the Xorg log, we can get the physical range of frame buffer. ... Linear framebuffer at 0xF0000000 ...

9). Search "0xF0000000" in the page table entries. Got these.

...

[722] va=b4800000 pte=f00004f7: pfn=0xf0000 noconsist largepage mod ref user write uncached

[723] va=b4c00000 pte=f04004f7: pfn=0xf0400 noconsist largepage mod ref user write uncached

[724] va=b5000000 pte=f08004f7: pfn=0xf0800 noconsist largepage mod ref user write uncached

[725] va=b5400000 pte=f0c004f7: pfn=0xf0c00 noconsist largepage mod ref user write uncached

[745] va=ba400000 pte=f5c004f7: pfn=0xf5c00 noconsist largepage mod ref user write uncached

...

10).In solairs the PAT/PWT/PCD values are setup so that:

PAT & PWT -> Write Protected

PAT & PCD -> Write Combining PAT by itself (PWT == 0 && PCD ==0) fields uncac0hables (same as PCD == 1) 0x04f7 = 0x0000010011110111
               ^      ^^ 

               |      ||_____________________________________________PAT 0 should be 1??

               |      |____________________________PCD 1

               |________________PWT 0

   for large pages Bit 12: PAT Bit 4: PCD Bit 3: PWT Any thing wrong here?

中国足球 智商[Read More]

8.3.3. D3 State Restoring a function from D3 requires the operating system to reinitialize the function, beginning with, for the case of D3cold , restoring power to the device and initiating a PCI Bus Segment Reset. This is accomplished by either programming the hosting bus’s Originating Device to D0 or by other ACPI-type control methods. Full context must be restored to the function before it is capable of resuming normal operation. For example, reinitialization includes, but is not necessarily limited to, restoring the Base Address registers, re-enabling the I/O and memory spaces, re-enabling bus master capabilities, and unmasking any IRQs or PCI Interrupts as well as restoring the INT Line register. Furthermore, if the function has the DSI bit set, the operating system is required to execute whatever initialization code is necessary, either via the device driver’s initialization code or by executing POST.

a PMCS(Power Management Control/Status is defined in the PCI spec. Offset D4~D5 in PCI configuration space. There is a PCI tool in Solaris which is un-disclosed yet to read and write PCI registers. dump the register in Offset D4~D5: pcitool /pci\@0\,0 -d bdf=0.2.0,config -o d4 -s 1 -b 0x1
0F 0E 0D 0C 0B 0A 09 08 07 06 05 04 03 02 01 00
-----------------------------------------------
0x00000000000000d0: -- -- -- -- -- -- -- -- -- -- -- 00 -- -- -- --
the Xorg crashed:
Set the register to 0x3(D3):
pcitool /pci\@0\,0 -d bdf=0.2.0,config -w 0x3 -o d4 -s 1
Backtrace:
1: 0xc4b3d39f: /lib/libc.so.1:__sighndlr+0xf
2: 0xc4b3288b: /lib/libc.so.1:call_user_handler+0x2b8
** Signal 11 (SEGV)
3: 0xc47ab1f2: /usr/X11/lib/modules/drivers/i810_drv.so:PreInitCleanup+0x162
4: 0xc47ac721: /usr/X11/lib/modules/drivers/i810_drv.so:I830BIOSPreInit+0x13d1
5: 0x80d0f3a: /usr/X11/bin/i386/Xorg:InitOutput+0xd6a
6: 0x80abb0a: /usr/X11/bin/i386/Xorg:main+0x29a
7: 0x8085ab0: /usr/X11/bin/i386/Xorg:_start+0x80
Fatal server error:
Caught signal 11. Server aborting
The screen is freezing. Mouse can not move around and the screen have no change whatever I hit.
Set the register back to 0x(D0):
pcitool /pci\@0\,0 -d bdf=0.2.0,config -w 0 -o d4 -s 1
pkill -HUP dtlogin
the Xserver can start again.
From the test above, Xorg need some extra work to handle the situation when the graphics card is set to D3 state. Otherwise, it will crash immediately. And the graphics card can be re-initialized successfully by only the PMCS.

今天听老板说Q辞职了, 心中荡过一丝凉意.哦,又一个人走了. 本来工作六年, 见过太多的离合, 应该司空见惯了, 但是还是有一些伤感. 每个人都有自己的理想, 但现实和理想相差太远, 大多数人选择的是逃避. 等待着他们的是人生的下一个起点, 新的希望, 新的心情. 也许一个公司只是一个利益的集合, 永远成不了一个家. 有人选择了离开, 有人选择了留守. 我有时候也在想, "大浪淘沙",留下来的未必是金子, 也许就是一颗普普通通的沙子. 乱七八糟, 我还是祝愿Q能在新地方发现自己新的价值.

Yesterday, We attended a VT conference hosted by Novell In Beijing. "Virtualization" is a hot topic nowadays. However, it really surprise me when I found there was no vacant seat in the hall. As the usual, after the boring opening remarks, Intel, IBM, Dell boast their solutions for Virtualization. They listed below as the advantage of Virtualization. ie., 1. Server consolidation. 2. Service migration and scalability. 3. Security. Unlike other companys, The engineer from AMD gave us a technical presentation. They showed their roadmap to support full Virtualiztion. Currently, Intel and AMD only support CPU Virtualization, and IO virtulization is not supported yet. The good news is Intel's VTD is on the way. And AMD promises to deliver virtulizated IO chipsets in 2009, nested MMU in 2007. Furthrer, muti-core and virtualization are like a inherent pal. They also demonstrated their desirable muti-core plan. Maybe, several years later, "everything is virtualized" is not only a advertisement, but also a must requirement by our customers.