红联Linux门户
Linux帮助

python并行包pypar在64位机器上的安装

发布时间:2008-08-06 10:16:21来源:红联作者:loife
pypar是一个简单小巧的python包,可用于编写并行处理的python程序。两年前在32位机器上,安装很顺利。但近来机群环境变化,需要在64位的Linux系统上重新编译,需要一点小技巧:

本系统是运行在2 Intel CPU x 双核 x 双线程的Dell机器上的Linux:

引用:
“”“
[xxia@dell3 examples]$ uname -r
2.6.9-67.0.4.ELsmp
[xxia@dell3 examples]$ cat /etc/*release*
Fedora Core release 4 (Stentz) but with CentOS 4 kernel
LSB_VERSION="1.3"
Fedora Core release 4 (Stentz) but with CentOS 4 kernel
”“”


首先,安装MPI (lam-7.1.1)时要建立共享库:

引用:
“”“
[xxia@dell3 lam-7.1.1]$ ./configure --enable-shared --prefix=/home/xxia/usr && make && make install
”“”


否则,连接pypar时会通不过:

引用:
“”“
[xxia@dell3 pypar_1.9.3]$ python2.5 setup.py install
gcc -I/home/xxia/usr/include -pthread -L/home/xxia/usr/lib -llammpio -llamf77mpi -lmpi -llam -laio -laio -lutil -ldl

running install
running build
running build_py
running build_ext
building 'pypar.mpiext' extension
gcc -pthread -shared build/temp.linux-x86_64-2.5/mpiext.o -L/home/xxia/usr/lib -llammpio -llamf77mpi -lmpi -llam -laio -lutil -ldl -o build/lib.linux-x86_64-2.5/pypar/mpiext.so
/usr/bin/ld: /home/xxia/usr/lib/libmpi.a(abort.o): relocation R_X86_64_32 against `lam_mpi_comm_world' can not be used when making a shared object; recompile with -fPIC
/home/xxia/usr/lib/libmpi.a: could not read symbols: Bad value
collect2: ld returned 1 exit status
error: command 'gcc' failed with exit status 1
“”“


其次,在pypar 1.9.3的安装程序中,64位系统的参数写法也不正确:

引用:
“”“
if os.name == 'posix' and os.uname()[4] == 'x86_64':
#Extra flags for 64 bit architectures
#extra_compile_args = ' -fPIC -m64' #Valid for gcc
extra_compile_args = ' -fPIC -tp amd64' #Valid for pgcc
else:
extra_compile_args = None
”“”


由此会导致错误:

引用:
“”“
[xxia@dell3 pypar_1.9.3]$ python2.4 setup.py install
gcc -I/home/xxia/usr/include -pthread -L/home/xxia/usr/lib -llammpio -llamf77mpi -lmpi -llam -laio -laio -lutil -ldl

running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.4
creating build/lib.linux-x86_64-2.4/pypar
copying lib/pypar/pypar.py -> build/lib.linux-x86_64-2.4/pypar
copying lib/pypar/__init__.py -> build/lib.linux-x86_64-2.4/pypar
running build_ext
building 'pypar.mpiext' extension
creating build/temp.linux-x86_64-2.4
Traceback (most recent call last):
File "setup.py", line 161, in ?
extra_compile_args=extra_compile_args)]
File "/home/xxia/usr/lib/python2.4/distutils/core.py", line 149, in setup
dist.run_commands()
File "/home/xxia/usr/lib/python2.4/distutils/dist.py", line 946, in run_commands
self.run_command(cmd)
File "/home/xxia/usr/lib/python2.4/distutils/dist.py", line 966, in run_command
cmd_obj.run()
File "/home/xxia/usr/lib/python2.4/distutils/command/install.py", line 506, in run
self.run_command('build')
File "/home/xxia/usr/lib/python2.4/distutils/cmd.py", line 333, in run_command
self.distribution.run_command(command)
File "/home/xxia/usr/lib/python2.4/distutils/dist.py", line 966, in run_command
cmd_obj.run()
File "/home/xxia/usr/lib/python2.4/distutils/command/build.py", line 112, in run
self.run_command(cmd_name)
File "/home/xxia/usr/lib/python2.4/distutils/cmd.py", line 333, in run_command
self.distribution.run_command(command)
File "/home/xxia/usr/lib/python2.4/distutils/dist.py", line 966, in run_command
cmd_obj.run()
File "/home/xxia/usr/lib/python2.4/distutils/command/build_ext.py", line 279, in run
self.build_extensions()
File "/home/xxia/usr/lib/python2.4/distutils/command/build_ext.py", line 405, in build_extensions
self.build_extension(ext)
File "/home/xxia/usr/lib/python2.4/distutils/command/build_ext.py", line 470, in build_extension
depends=ext.depends)
File "/home/xxia/usr/lib/python2.4/distutils/ccompiler.py", line 699, in compile
self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
File "/home/xxia/usr/lib/python2.4/distutils/unixccompiler.py", line 112, in _compile
self.spawn(self.compiler_so + cc_args + [src, '-o', obj] +
TypeError: can only concatenate list (not "str") to list
[xxia@dell3 pypar_1.9.3]$
”“”


正确的代码应是:

引用:
“”“
if os.name == 'posix' and os.uname()[4] == 'x86_64':
#Extra flags for 64 bit architectures
#extra_compile_args = ' -fPIC -m64' #Valid for gcc
extra_compile_args = ' -fPIC -tp amd64' #Valid for pgcc

extra_compile_args = [' -fPIC -m64'] #added by xxq
#extra_compile_args = None #added by xxq
#extra_compile_args = [' -fPIC -tp -m64'] #added by xxq
else:
extra_compile_args = None
”“”


即便这们,仍有错误:

引用:
“”“[xxia@dell3 pypar_1.9.3]$ python2.5 setup.py install
gcc -I/home/xxia/usr/include -pthread -L/home/xxia/usr/lib -llammpio -llamf77mpi -lmpi -llam -laio -laio -lutil -ldl

running install
running build
running build_py
running build_ext
building 'pypar.mpiext' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/home/xxia/usr/include -I/home/xxia/usr/include/python2.5 -c mpiext.c -o build/temp.linux-x86_64-2.5/mpiext.o -fPIC -m64
gcc: mpiext.c: No such file or directory
gcc: -fPIC -m64: No such file or directory
gcc: no input files
error: command 'gcc' failed with exit status 1
[xxia@dell3 pypar_1.9.3]$
“”“


解决办法是将参数 "-fPIC -m64"前置,手工编译:

引用:
”“”
[xxia@dell3 pypar_1.9.3]$ gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/home/xxia/usr/include -I/home/xxia/usr/include/python2.5 -c mpiext.c -fPIC -m64 -o build/temp.linux-x86_64-2.5/mpiext.o
“”“


然后再安装:

引用:
“”“[xxia@dell3 pypar_1.9.3]$ python2.5 setup.py install
gcc -I/home/xxia/usr/include -pthread -L/home/xxia/usr/lib -llammpio -llamf77mpi -lmpi -llam -laio -laio -lutil -ldl

running install
running build
running build_py
running build_ext
building 'pypar.mpiext' extension
gcc -pthread -shared build/temp.linux-x86_64-2.5/mpiext.o -L/home/xxia/usr/lib -llammpio -llamf77mpi -lmpi -llam -laio -lutil -ldl -o build/lib.linux-x86_64-2.5/pypar/mpiext.so
running install_lib
creating /home/xxia/usr/lib/python2.5/site-packages/pypar
copying build/lib.linux-x86_64-2.5/pypar/pypar.py -> /home/xxia/usr/lib/python2.5/site-packages/pypar
copying build/lib.linux-x86_64-2.5/pypar/__init__.py -> /home/xxia/usr/lib/python2.5/site-packages/pypar
copying build/lib.linux-x86_64-2.5/pypar/mpiext.so -> /home/xxia/usr/lib/python2.5/site-packages/pypar
byte-compiling /home/xxia/usr/lib/python2.5/site-packages/pypar/pypar.py to pypar.pyc
byte-compiling /home/xxia/usr/lib/python2.5/site-packages/pypar/__init__.py to __init__.pyc
running install_egg_info
Writing /home/xxia/usr/lib/python2.5/site-packages/Pypar-1.9.2-py2.5.egg-info
“”“


至此,大功乃成。
文章评论

共有 3 条评论

  1. liuchunxiao0611 于 2014-08-06 18:22:16发表:

    学习学习

  2. 于 2013-12-19 23:48:37发表:

    请问如何在windows下python2.7下安装pypar呢!求详细步骤。。。谢谢

  3. 于 2013-12-19 23:45:50发表:

    请问在32机器上要怎么安装啊?