[Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite

[Tim Peters]

[Tim]
>> $9 = {_ob_next = 0x0, _ob_prev = 0x0, ob_refcnt = 0,
>>       ob_type = 0x8130160, length = 1, str = 0x41903130,
>>       hash = -1, defenc = 0x0}
>>
>> that certainly means the object has been released already.

[Jeremy]
> This is code that is trying to revive an object from the freelist, so
> it isn't a big surprise that the object looks like it has been freed.

Doh -- yup, of course.

> Now here's a very strange thing.  If I apply the following patch, the
> problem goes away.  The patch doesn't make a lot of sense to me, so
> I'm guessing that it's just a lucky patch -- avoiding the problem
> without fixing the cause.

We should move this to python-dev.  OK, I just did <wink>.  Another clue
about the cause coming up.

> The thing that is curious is that we change the test against
> unicode->str[0], which is probably an unsigned int,

Why?  Are you doing a UCS4 build?  The expansion of Py_UNICODE (unicode->str
is an array of Py_UNICODE thingies) can't be guessed from where I sit (it
expands to "unsigned short" on Windows, but that's because that's hardcoded
in PC\pyconfig.h; I'm not sure where you get its expansion from).

> so that we also test that it is > 0.  In the specific case that
> crashes, the debugger says the value is -875836469.

That's probably a great clus!  In hex, that's 0xcbcbcbcb, which is the
special byte pattern the debug pymalloc sprays into freshly-allocated
memory.  IOW, unicode->str contains (in a release build) uninitialized heap
trash at this point, and so there's nothing it can be tested against that
makes any sense.  This suggests a deeper bug in the Unicode support.

> Does that make any sense?  Do we actually have a signed value here?

You have to figure out what Py_UNICODE expands to in your build to answer
that one.

> Or does it get converted to a signed value when the comparison occurs?

If gdb showed you -875836469 in response to "unicode->str[0]" in isolation,
I have to guess Py_UNICODE is expanding to a signed 4-byte type in your
build.

> Jeremy
>
> Index: unicodeobject.c
> ===================================================================
> RCS file: /cvsroot/python/python/dist/src/Objects/unicodeobject.c,v
> retrieving revision 2.196
> diff -c -r2.196 unicodeobject.c
> *** unicodeobject.c	16 Sep 2003 03:41:45 -0000	2.196
> --- unicodeobject.c	16 Sep 2003 03:55:53 -0000
> ***************
> *** 130,138 ****
>       /* Resizing shared object (unicode_empty or single character
>          objects) in-place is not allowed. Use PyUnicode_Resize()
>          instead ! */
>       if (unicode == unicode_empty ||
>   	(unicode->length == 1 &&
> ! 	 unicode->str[0] < 256 &&
>   	 unicode_latin1[unicode->str[0]] == unicode)) {
>           PyErr_SetString(PyExc_SystemError,
>                           "can't resize shared unicode objects");
> --- 130,142 ----
>       /* Resizing shared object (unicode_empty or single character
>          objects) in-place is not allowed. Use PyUnicode_Resize()
>          instead ! */
>       if (unicode == unicode_empty ||
>   	(unicode->length == 1 &&
> ! 	 unicode->str[0] < 256 && unicode->str[0] > 0 &&
>   	 unicode_latin1[unicode->str[0]] == unicode)) {
>           PyErr_SetString(PyExc_SystemError,
>                           "can't resize shared unicode objects");

Belaboring the obvious, if unicode->str[0] is really heap trash at this
point, there's no safe test.