Thread: [Bashdb-pydb] pydb-1.21 buglets

Some tweaks to pydb.el for better windows support (c: and spaces in
filenames).  BTW is pydb-pydbtrack-stack-entry-regexp currently used for
anything?

--8<---------------cut here---------------start------------->8---
(setq gud-pydb-marker-regexp
      "^(\\(\\(?:[a-zA-Z]:\\)?[-a-zA-Z0-9_/.\\\\ ]+\\):\\([0-9]+\\))")
(setq pydb-position-re
      "\\(^\\|\n\\)(\\(\\(?:[A-Za-z]:\\)?[^:]+\\):\\([0-9]*\\)).*\n")
(setq pydb-pydbtrack-stack-entry-regexp  
      "^(\\((?:[a-zA-Z]:\\)?[-a-zA-Z0-9_/.\\\\ ]*\\):\\([0-9]+\\)):[ \t]?\\(.*\n\\)")))
--8<---------------cut here---------------end--------------->8---

A couple of small python changes:

* raw_input() needs a strip() because when running pydb with ipython and
emacs, I keep getting a trailing CR, which breaks things.
* pydbcmd.py: 419
looks like an indentation error. The line trace was *always* being printed.

Thanks for the patches but I'm having trouble using some of them. 

DaveS writes:
 > 
 > Some tweaks to pydb.el for better windows support (c: and spaces in
 > filenames).  BTW is pydb-pydbtrack-stack-entry-regexp currently used for
 > anything?

Perhaps not. I've removed for now. We'll see what breaks if anything.

 > 
 > --8<---------------cut here---------------start------------->8---
 > (setq gud-pydb-marker-regexp
 >       "^(\\(\\(?:[a-zA-Z]:\\)?[-a-zA-Z0-9_/.\\\\ ]+\\):\\([0-9]+\\))")
 > (setq pydb-position-re
 >       "\\(^\\|\n\\)(\\(\\(?:[A-Za-z]:\\)?[^:]+\\):\\([0-9]*\\)).*\n")
 > (setq pydb-pydbtrack-stack-entry-regexp  
 >       "^(\\((?:[a-zA-Z]:\\)?[-a-zA-Z0-9_/.\\\\ ]*\\):\\([0-9]+\\)):[ \t]?\\(.*\n\\)")))
 > --8<---------------cut here---------------end--------------->8---

Please give a diff against the current emacs/pydb.el for this. And
also please add a test in emacs/pydb-test.el.in for the a sample
string this now matches. I'm not sure I could reliably make a change
from what you have above.

 > 
 > A couple of small python changes:
 > 
 > * raw_input() needs a strip() because when running pydb with ipython and
 > emacs, I keep getting a trailing CR, which breaks things.

This change has been applied in CVS.

 > * pydbcmd.py: 419
 > looks like an indentation error. The line trace was *always* being printed.


No, this is intentional. Look at gdb output and you'll see a similar
output format.

 > 
 > Index: C:/Python25/Lib/site-packages/pydb/gdb.py
 > ===================================================================
 > --- C:/Python25/Lib/site-packages/pydb/gdb.py	(revision 1)
 > +++ C:/Python25/Lib/site-packages/pydb/gdb.py	(revision 5)

As mentioned above, I've applied this change.  It doesn't break the
regression tests. Presumably if it breaks things for someone else
we'll find out down the line.

But in the future place patch against the source (which in this case
the basename of the file is gdb.py.in)


 > One last change I didn't include in the patch (because it's so ugly).
 > traceback.print_exc() prints to stderr by default, but when I'm running in
 > emacs that seems to get buffered so I don't see the traceback until I exit
 > pydb.  For now I'm just calling as traceback.print_exc(file=p.stdout) to
 > make it work.  Still looking for a better alternative.

p.stdout seems better than stderr; we allow all debugger output to get
redirected to wherever was specified. If it's not doing that it's a
bug. I've just put this change also in CVS.

 > 
 > Somethings changed with the tracebacks.  When execution stops because of an
 > exception, I'm getting dropped deep into pydb code rather then at the point
 > of failure.  Example:

Yes, this has changed. On an exception in Python 2.5 you are now at
the point at which the exception was fielded rather than at the point
it was raised. Previously there was no way to know, and personally I
think this is a little more accurate. However in the future I may
consider adding some option to allow the old behavior. For now if you
always want to be at the the place the exception was raised issue
"frame 0".

Lastly, expect alas more breakage with pydb, ipython, emacs. Recently
I've been changing the first two so that colorized output
appears. (See the ipython forum on that. There are some patches I
haven't sent in yet though.) However the recent changes seems to have
cause the line-tracking regexp to now fail.

On Sun, Feb 18 2007, R. Bernstein wrote:

> Thanks for the patches but I'm having trouble using some of them. 
>
> DaveS writes:
>  > 
>  > Some tweaks to pydb.el for better windows support (c: and spaces in
>  > filenames).  BTW is pydb-pydbtrack-stack-entry-regexp currently used for
>  > anything?
>
> Perhaps not. I've removed for now. We'll see what breaks if anything.
>
>  > 
>  > --8<---------------cut here---------------start------------->8---
>  > (setq gud-pydb-marker-regexp
>  >       "^(\\(\\(?:[a-zA-Z]:\\)?[-a-zA-Z0-9_/.\\\\ ]+\\):\\([0-9]+\\))")
>  > (setq pydb-position-re
>  >       "\\(^\\|\n\\)(\\(\\(?:[A-Za-z]:\\)?[^:]+\\):\\([0-9]*\\)).*\n")
>  > (setq pydb-pydbtrack-stack-entry-regexp  
>  >       "^(\\((?:[a-zA-Z]:\\)?[-a-zA-Z0-9_/.\\\\ ]*\\):\\([0-9]+\\)):[ \t]?\\(.*\n\\)")))
>  > --8<---------------cut here---------------end--------------->8---
>
> Please give a diff against the current emacs/pydb.el for this. And
> also please add a test in emacs/pydb-test.el.in for the a sample
> string this now matches. I'm not sure I could reliably make a change
> from what you have above.
>

I'll take a crack at this tomorrow.

>  > 
>  > A couple of small python changes:
>  > 
>  > * raw_input() needs a strip() because when running pydb with ipython and
>  > emacs, I keep getting a trailing CR, which breaks things.
>
> This change has been applied in CVS.
>
>  > * pydbcmd.py: 419
>  > looks like an indentation error. The line trace was *always* being printed.
>
>
> No, this is intentional. Look at gdb output and you'll see a similar
> output format.
>

Ah, didn't realize that.  I guess the only issue with that is if pydb is not
running through gud (say in ipython) you get a double tracer - the first
(source:line) output that gdb would hide if it was running plus this second
tracer.  I'll get use to it.

>  > 
>  > Index: C:/Python25/Lib/site-packages/pydb/gdb.py
>  > =================================================================== ---
>  > C:/Python25/Lib/site-packages/pydb/gdb.py (revision 1)
>  > +++ C:/Python25/Lib/site-packages/pydb/gdb.py	(revision 5)
>
> As mentioned above, I've applied this change.  It doesn't break the
> regression tests. Presumably if it breaks things for someone else
> we'll find out down the line.
>
> But in the future place patch against the source (which in this case
> the basename of the file is gdb.py.in)
>

Sorry about that, I've never really worked with configure before and I just
kind of forgot about those .in files.  I'll need to adjust my work patterns
a bit.

>
>  > One last change I didn't include in the patch (because it's so ugly).
>  > traceback.print_exc() prints to stderr by default, but when I'm running in
>  > emacs that seems to get buffered so I don't see the traceback until I exit
>  > pydb.  For now I'm just calling as traceback.print_exc(file=p.stdout) to
>  > make it work.  Still looking for a better alternative.
>
> p.stdout seems better than stderr; we allow all debugger output to get
> redirected to wherever was specified. If it's not doing that it's a
> bug. I've just put this change also in CVS.
>

Does this mean that all output should make it's way to the debugger stdout
member?  If so I'll keep my eye open for any other cases.

>  > 
>  > Somethings changed with the tracebacks.  When execution stops because
>  > of an exception, I'm getting dropped deep into pydb code rather then at
>  > the point of failure.  Example:
>
> Yes, this has changed. On an exception in Python 2.5 you are now at
> the point at which the exception was fielded rather than at the point
> it was raised. Previously there was no way to know, and personally I
> think this is a little more accurate. However in the future I may
> consider adding some option to allow the old behavior. For now if you
> always want to be at the the place the exception was raised issue
> "frame 0".

I guess this is more accurate, but if I'm working on a script I don't
normally need to see all the stack frames from the debugging framework.  I
like to think that my script is the only thing running.  I vote +1 for being
able to hide, or at least easily ignore, the pydb frames.

>
> Lastly, expect alas more breakage with pydb, ipython, emacs. Recently I've
>been changing the first two so that colorized output appears. (See the
>ipython forum on that. There are some patches I haven't sent in yet
>though.) However the recent changes seems to have cause the line-tracking
>regexp to now fail.
>

I've previously seen some issues related to coloured text.  If the
ansi-codes aren't processed before the tracking filters scan the text the
regexps can fail - and some filters work with the passed text while others
directly scan the buffer contents, so it's a bit of a hair puller at times.

Looking forward to more goodness.  

-- 
DS

DaveS writes:
 > Ah, didn't realize that.  I guess the only issue with that is if pydb is not
 > running through gud (say in ipython) you get a double tracer - the first
 > (source:line) output that gdb would hide if it was running plus this second
 > tracer.  I'll get use to it.

Actually, it is *more* helpful when not run under Emacs and in ipython
in particular. Try running %pydb and stepping lines. It is helpful to
see the text of source shown.

Furthermore, in pydb's CVS I've broken out and the exposed the method
that prints that source line. This was done for ipython in particular,
but it could be helpful for other things of that ilk, e.g. remote
execution or Eclipse which may want results tagged in XML.

In ipython, I've patched Debugger.py to wrap a PyColorize call to that
source line. Recall that ipython creates a customized Pdb object
suited for its particular way of doing things. The patch hasn't been
submitted because there are already a number of patches in ipython
that I've sent in that haven't been applied -- and some of those need
to get in for this to work.

However it's also possible for ipython or some other such front-end to
customize an enclosing method, like "print_location". And there it
could choose not to show that line. (But again for ipython in
particular, I don't think it advisable when run outside of Emacs.)

 > > p.stdout seems better than stderr; we allow all debugger output to get
 > > redirected to wherever was specified. If it's not doing that it's a
 > > bug. I've just put this change also in CVS.
 > >
 > 
 > Does this mean that all output should make it's way to the debugger stdout
 > member?  If so I'll keep my eye open for any other cases.

Yes - thanks. For example in version 1.21 a change was made so that
disassembly output no longer goes to stdout unconditionally.

 > > Yes, this has changed. On an exception in Python 2.5 you are now at
 > > the point at which the exception was fielded rather than at the point
 > > it was raised. Previously there was no way to know, and personally I
 > > think this is a little more accurate. However in the future I may
 > > consider adding some option to allow the old behavior. For now if you
 > > always want to be at the the place the exception was raised issue
 > > "frame 0".
 > 
 > I guess this is more accurate, but if I'm working on a script I don't
 > normally need to see all the stack frames from the debugging framework.  I
 > like to think that my script is the only thing running.  I vote +1 for being
 > able to hide, or at least easily ignore, the pydb frames.

Yes, this has crossed my mind on a number of occasions. And in fact we
do some of this already on the most-recent side of the stack when you
use the "--threading" option. What you are suggesting is hiding frames
at the other end. (pdb has this problem too which is where we
inherited the bad behavior.)

I'll also note that the mechanism for doing this is a little
cumbersome and error prone. Either you look for certain source file
names like "bdb.py" (error prone), or you create a variable in the
call stack that you introspect on and identify like "breadcrumb" which
has a value of the method it is in that you can compare against
(cumbersome). And this latter alternative isn't available if the code
comes from outside of pydb as bdb.py does.

 > > Lastly, expect alas more breakage with pydb, ipython, emacs. Recently I've
 > >been changing the first two so that colorized output appears. (See the
 > >ipython forum on that. There are some patches I haven't sent in yet
 > >though.) However the recent changes seems to have cause the line-tracking
 > >regexp to now fail.
 > 
 > I've previously seen some issues related to coloured text.  If the
 > ansi-codes aren't processed before the tracking filters scan the text the
 > regexps can fail - and some filters work with the passed text while others
 > directly scan the buffer contents, so it's a bit of a hair puller at times.

Yes, so help here would be appreciated. Although my preferred
interaction of pydb is via Emacs, I don't really use ipython inside
emacs. I'll often just use a the Emacs pydb debugger, or a regular
shell with pydb line tracking, or a python shell.

But tolerance for other modes is good. I've noticed that many of the
fixes needed for such things are generally useful to make pydb better
or more general.

There is another gdb-like debugger for Ruby (called ruby-debug). It's
overall design is a bit cleaner. In particular, the entire "printing"
mechanism is its own class. Such a structure like this might be of
help here.

 > Looking forward to more goodness.  

I can only go so far and have limited time I can spend on this. So
really a lot depends on everyone or others.