Building Python-3.1 with sqlite3 and SSL support on NT 4-优快云博客

I am building Python 3.1.1 again. Last time I did it was in Year 2009. This time I am building it again, and write down the steps I take. Besides, I will try to build sqlite support for it.

The first a few steps:

Read README. It tells me to read PCbuild\readme.txt.
Read PCbuild\readme.txt. It tells me that Visual Studio 2008 is required to build the project. And that's why the resulting python can't run on NT 4. I have to read further. It tells me to go to PC\VC6 for VC6 support.
Make sure Visual Source Safe is not installed (otherwise you'll be prompted the VSS connection for each and every project). Open pcbuild.dsw. A list of projects will be loaded in Visual C++ 6.0 (VC98).
Directly build (menu item Build -> Rebuild All). There are errors.

Now we go to investigate how to remove the errors. Firstly, set active configuration to _ctypes Release (Build -> Set Active Configuration). This is because we only want the Release build. I recall that I had a text file describing how to do this:

Programming\Documents\Languages\Python\Building Python 3.1 for NT4.txt

That file apparrently mentioned these important points regarding build errors:

ULONG_PTR is not defined in VC6. We need to define it. We can define it in WINDEF.H, right after the definition of DWORD, as "typedef DWORD ULONG_PTR;".
In _ctypes.dsp, INVALID_FILE_ATTRIBUTES is not defined. It is the return value of GetFileAttributes. The value should be 0xFFFFFFFF. By right-clicking FILE_ATTRIBUTE_DIRECTORY and go to its definition, you can see that it is defined in WINNT.H. We then define INVALID_FILE_ATTRIBUTES afterwards.

After that, there is no error but 20 warnings. They are of different reasons. Even in the previous "Building" text file, I didn't analyze all the detailed situations.

Then we can repeatedly set active configuration and build all other projects. Or you can try right-clicking the project (in FileView) and build it (make sure the release configuration is used).

The list of modules that can't be built include (ignore the list in my previous "Building" text file):

_msi
_sqlite3
_ssl
_tkinter
_bz2

All other modules can pass compilation. Next we'll look at how to build sqlite3.

Firstly, create a new project for sqlite3. Right-click the workspace in FileView, and select Add New Project. Create a Win32 Dynamic-Link Library, with project name sqlite3_dll. Select "Dependency of" and "_sqlite3". I downloaded the 3.7.17 version of sqlite3 to have a try. The code is an amalgamation, which means all code is put into one C file, sqlite3.c.

Add sqlite3.c to the project, and sqlite3.h. Build in release. No error, but 21 warnings. As I checked several warnings, I can't find anything critical. Let's just ignore these warnings for now.

Now change the project settings to make the output Release\sqlite3.dll instead of the default Release\sqlite3_dll.dll. This can be done on the Link tab, General category.

Now change the project settings of _sqlite3. In C/C++ tab, Preprocessor category, there is a Additional include directory field. Replace ..\..\..\sqlite-source-3.3.4 with ..\..\..\sqlite3\sqlite-amalgamation-3071700 (where you extract the SQLite source code). Build shows one error: import library sqlite3.lib is missing.

I found the def file needs to be retrieved from the pre-built win32 dll of SQLite by reading

http://eli.thegreenplace.net/2009/09/23/compiling-sqlite-on-windows/

On the other hand, I can try to mark the export functions by myself in the sqlite3.h with _declspec( dllexport ), but I don'texactly know which are external ones should be exported, so let's take the blog's way.

1. Add the def file to the project. It automatically appears in the Source Files.
2. The blog mentions to add preprocessor definition THREADSAFE (all definitions without explicit value default to "1" without quotes). Actually this definition is optional, because by default, SQLITE_THREADSAFE is 1.
2.1 This flag being 1 means that the library will be thread-safe. More comments can be found in sqlite3.c regarding SQLITE_THREADSAFE values 0, 1 and 2 (it copies the value from THREADSAFE). 0 or 2 may bring higher performance, but the hosting applications must guarantee the access to the database happens only from one thread (for details, read the comments in the source code). Note that the behavior of definition default to 1 is specific to Visual C++. Borland C++ Builder however, defaults the value to a null string. To be explicity, you can use THREADSAFE=1 in the C/C++ tab Preprocessor category. You can use below preprocessor statements to test whether the value is 1:

 #   define SQLITE_THREADSAFE THREADSAFE
+#       if SQLITE_THREADSAFE == 1
+        aaarobbiebomb
+#       endif
 # else

3. Build shows 10 errors, such as "sqlite3.def : error LNK2001: unresolved external symbol sqlite3_column_database_name". The blog mentions definition SQLITE_ENABLE_COLUMN_METADATA is needed to make it compile cleanly (this definition needn't have any explicit value). In my build, SQLITE_ENABLE_RTREE is required, too, otherwise sqlite3_rtree_geometry_callback would not be found. Now build passes.

We should now edit the Link tab General category in _sqlite3 project settings. Change the path ..\..\..\sqlite-source-3.3.4\sqlite3.lib to sqlite3_dll\Release\sqlite3.lib. Now we should be able to build.

Let's copy out files from Python-3.1.1\PC\VC6 to the installation directory of Python 3.1, which is typically C:\Python31. Make sure you have backed up all the old files because we'll overwrite them soon. Go! exe files go to Python31. pyd files and sqlite3.dll go to Python31\DLLs. python31.dll to C:\WINNT\system32\.

_sqlite3.pyd and sqlite3.dll are the two output files of sqlite3 support.

Python 3.1 starts and plays a little bit with seems fine. Now I want to run some pysqlite tests. I directly run types.py but it shows below error:

Traceback (most recent call last):
  File "types.py", line 24, in <module>
    import unittest
  File "C:\Python31\lib\unittest.py", line 56, in <module>
    import types
  File "C:\WINNT\Profiles\Administrator\Personal\test2\types.py", line 30, in <m
odule>
    class SqliteTypeTests(unittest.TestCase):
AttributeError: 'module' object has no attribute 'TestCase'

Later I found it's because of the name conflict of types.py with the default types.py. After renaming the types.py file (to sqltype.py), tests can run successfully.

Up to now, cherrypy is not directly supported, since SSL is not yet compiled.

== 2013-08-17 Building OpenSSL ==

I pulled the OpenSSL subversion by following Python's PCbuild\readme.txt: svn exporthttp://svn.python.org/projects/external/openssl-0.9.8g

Python documentation mentioned that building OpenSSL not only requires a C++ compiler, but also NASM (Netwide Assembler, an open source software). As mentioned in OpenSSL INSTALL.W32, ActivePerl is also needed. However, I'm worried that the current version of ActivePerl no longer supports NT 4. Let's try other methods if any obstacle is met. I installed NASM 2.09.10, which I had previously.

As I tried, indeed ActivePerl 5.16.3 (the current version) doesn't support NT 4, that it reports "Cannot find Process32Next in kernel32.dll". Fortunately, the lowest version I found download is ActivePerl-5.6.1.638-MSWin32-x86.msi, and it works on NT 4.

Skip below part because it is not really necessary
{
1. Run perl Configure like:

perl Configure VC-WIN32 --prefix=C:\WINNT\Profiles\Administrator\Personal\openssl-inst

The prefix switch specifies the installation target.
2. Run below command. No error occurs from my side.

ms\do_nasm.bat

3. In Visual C++ command prompt (or in an environment where the necessary environment variables are set, such as after calling "%SystemDrive%\Program Files\Microsoft Visual Studio\VC98\Bin\vcvars32.bat", run:

nmake -f ms\ntdll.mak

. . . but a warning of invalid escape sequence blocked the instruction.
}

After reading the comments in build_ssl.py, I made the compilation work with below commands:

cd Python-3.1.1\PCbuild
"%SystemDrive%\Program Files\Microsoft Visual Studio\VC98\Bin\vcvars32.bat"
path C:\Program Files\nasm;%PATH%
copy "C:\Program Files\nasm\nasm.exe" "C:\Program Files\nasm\nasmw.exe"
python.exe build_ssl.py Release Win32

Then, go to the Visual C++ 6 project, and build _ssl. It should now work, because the lib files exist in the out32 directory in the openssl directory. Copy _ssl.pyd to the Python31\DLLs directory.