100 pushups - DONE

OK, it took me about a year and a half. And they weren't pretty. But I finished it.

Sort of anti-climactic somehow after all this time. But kinda cool to have gotten there.

2 questions:
- do I bother maintaining this level? (once a week refresher?)
- what should I work on next? (chin ups probably)

100 pushups milestone - 80

OK, so I'm 80% of the way there (100 pushups).

Just checked my records and I've been doing this for 1 year and 3 months. That would be funny if it weren't so tragic. How can it be taking so long? Oh well, keep on trucking.

I better get a damn spiffy prize when I finish this.

Git 101 tutorial

http://en.oreilly.com/oscon2009/public/schedule/detail/7953

This was a great tutorial. It was as good of a brain dump as you could hope for in a half day session. A good mix of theory and hands on examples. This got me excited to get back to the office and push to get this adopted as soon as possible. (It's already "in the air". It just needs a nudge to get us converted over to it)

Well done!

Still Looking for the Swan in Squeak's Ugly Duckling

http://en.oreilly.com/oscon2009/public/schedule/detail/8158

So this is not exactly the tutorial I was expecting. But it's also quite possible my expectations were a little skewed.

Unfortunately Avi Bryant wasn't able to make the conference so Randal Schwartz stepped in. And no disrespect to Randal but if I had known this substitution had occurred, I would have picked a different tutorial simply because I had seen his presentation last year and was interested in a different perspective.

Partly I feel like I was expecting too much from this tutorial and part of me feels like I got something different than described.

When I read the description in the link above I don't get the sense of a remedial no-background talk. But like I said Randal took this over at the last minute for Avi so I have no complaints with him.

I think I focused a little too much on the last paragraph of the description whereas I should realize that that is what usually gets covered least at the end (or not at all):

"But we’ll also address the practical concerns that keep people away from Squeak: how to get rid of the pastel colors and bitmapped fonts so that you can stand to look at it; how to get your source code into version control so you can collaborate with others; how to find documentation and examples; how to integrate with the OS and with C libraries; how to manage deployment."

I am relatively comfortable with the ideas of smalltalk and reading it's code, just not completely sold on the language and environment. So what I was mostly looking for was the addressing of "practical concerns that keep people away from Squeak":

• how to get rid of the pastel colors and bitmapped fonts so that you can stand to look at it;
  
  • just a mention that this is possible. I would have really liked a detailed course on how to customize the ugly duckling away
  
  
• how to get your source code into version control so you can collaborate with others;
  
  • there was a pretty good description of version control and options for doing this
  
  
• how to find documentation and examples;
  
  • this was well done and is a core part of the wonder of smalltalk
  
  
• how to integrate with the OS and with C libraries;
  
  • I don't think there was any mention of this (unless I really zoned out)
  
  
• how to manage deployment.
  
  • I don't recall anything like this
  
  

But really it was a well done tutorial, just not what I was expecting. But I think
this was just a combination of over ambitious expectations and last minute teacher changes.

Here is the smalltalk course I would love to have:

• develop a real world useful app while I watch. e.g. take some unix sysadmin tasks and automate them and create a reporting system etc. ie, show me that squeak can kick python's ass at something where python excels
  
• show me how to recover when my image crashes or I've accidentally broken things
  
• show me how to customize my way from the default image to one of the premade developer images. then explain to me why these aren't already the defaults
  
• show me how to convince my bosses that I should do a trial project in smalltalk. :)
  

Happy Birthday - My mission to get to 100 pushups is now one year old!

So after a year of the 100 pushup plan, where am I? 75.

Now if you are familiar with the 6 weeks to 100 pushup plan you might detect a slight disconnect between the plan and my success. Just an eentsy bit off...

On the one hand: holy crap I can do 75 pushups. And also: wow, look at the determination and will power.

On the other hand: what is wrong with me? Why can't I finish this sucker? And: why am I wasting my time, focus on this?

So there you go. I'm part super man and part loser. I knew that before I started.

OK. If I haven't gotten to 100 before another year passes I seriously have to reevaluate this whole thing.

(Any tips from the pros out there how to get the last 25?)

One New Language a Year: (was) Smalltalk

OK, so I had this crazy plan of doing the "one new language a year" thing. I really like the idea of this. Get out of your comfort zone, prevent settling for a "blub" language, learn new techniques. So I choose smalltalk and thought this would fit the bill. I was really excited to work with turtles all the way down. "Real" object oriented programming. Etc.

And here it is May and I'm just not doing so well. Best laid schemes and all that. In fact I've decided to abandon it for, get this, emacslisp. Only time will tell if this was a good decision or just my normal fickleness.

Part of this decision is just pure pragmatism. My learning windows for this plan are of necessity going to be early in the morning before breakfast or late at night after the family's in bed. I've tried both slots for a month or more and both put me to sleep. You can't learn much while you are asleep so no matter the plan/desire I have to be doing something that engages my attention.

OK, I said it. Smalltalk puts me to sleep. And I feel bad for saying it. I really want to be the guy who likes smalltalk and wields it with authority. But there is just so much I don't enjoy about it and while I'm sure there are some cool things to learn from it, it just doesn't entice me. At least not at 6 am. We'll see if emacslisp can do better.

Smalltalk is like this quirky, fairly attractive girl who on paper seems like a good match for you ("Wow, you like continuations? Me too!") but for whatever reason just doesn't ignite that spark. In addition, she has lots of eccentricities that if she was "the one" would be charming/braggable. But if you are indifferent to her they just come off as irritations/oddities.

I also wonder if watching this video put the final nail in the coffin: "what killed smalltalk" (also). Just hearing about smalltalk being dead (I'm not defending this thesis) sort of activated my pragmatism module which some how grabbed the reins of my brain and executed veto power.

The thing that bugs me is that I don't see myself as a practical dude. In fact, I find the thought of being practical a little bit horrifying. I'm always doing things just because they are interesting, not because I'm trying to be more efficient, etc. And here I am making a practical decision. Ugg.

I wonder what the rules are about abandoning your "one new language a year" language halfway through the year. Is there some governing body I should contact? Do I need to ask permission first?

For what it's worth, smalltalk is not permanently abandoned; it is just pushed down on the list. If I stick with the language a year program I will almost certainly head back to smalltalk at some point. We'll see what happens next time around.

Does working with python as your primary language makes it harder to learn new languages? Every language that I've learned so far has been a productivity boost or has some other sweetener (my language history: basic -> pascal -> c -> c++ -> java -> perl -> python). emacslisp is a booster just because it is the only option in emacs (I will be looking into pymacs though). haskell looks like it might be a next step in the awesomeness hierarchy. But smalltalk doesn't *seem* (in my limited experience) like a significantly more productive tool past python. smalltalk feels like a language from which many great ideas have been lifted. And the ones that haven't yet are either in the process of being stolen (pypy - turtles all the way down) or aren't worth the trouble (image files, etc).

What's funny is that I've talked to others who have had the same initial fascination with smalltalk and then given up on it due various pain points or lack or practicality.

I guess in the end part of it is that smalltalk doesn't seem to solve a problem that I'm having in a vastly superior way and (surprisingly to me) isn't more fun. On paper it seems it should be. And I actually feel guilty for not liking smalltalk. I've failed the gods of language geekdom.

Any way, on to the new hotness of, er, emacslisp.

So why emacslisp? At the start of the year I was trying to decide between smalltalk and haskell. I was in sort of an object-y mood at the time so smalltalk won out. I'm not defaulting to haskell at this time primarily because I want to give it the full year (and it just feels like it has to be a calendar year starting in January). emacslisp works because I know it will be useful. It will make a real difference to my productivity immediately (I mostly live in emacs) and I like lispy languages (though I've only dabbled and/or used them for a semester in an ai class). *And* I don't feel too bad about giving it only half a year.

My goal with emasclisp is to get to the point where I can program in it as easily as I think (how it is for python with me now). I want to be able to whip out useful helper functions like this and be able to write my own modes and/or hack on existing emacs packages.

Any way, smalltalk, it's not you it's me. When I get my head right maybe we'll meet again and give it another shot. I wish you all the best. (but please don't call - I'm not sure emacslisp would understand us "just being friends").

Higher Order Perl (Python Style) : Chapter 7 - Higher Order Functions And Currying

TOC

### 7.1 Currying


# @tagged_texts = walk_html($tree, sub { ['MAYBE', $_[0]] },
#                                  \&promote_if_h1tag);
# sub promote_if_h1tag {
#   my $element = shift;
#   if ($element->{_tag} eq 'h1') {
#     return ['KEEPER', join '', map {$_->[1]} @_];
#   } else {
#     return @_;
#   }
# }
# sub extract_headers {
#   my $tree = shift;
#   my @tagged_texts = walk_html($tree, sub { ['MAYBE', $_[0]] },
#                                       \&promote_if_h1tag);
#   my @keepers = grep { $_->[0] eq 'KEEPER' } @tagged_texts;
#   my @keeper_text = map { $_->[1] } @keepers;
#   my $header_text = join '', @keeper_text;
#   return $header_text;
# }
def extender(accum, item):
    accum.extend(item)
tagged_texts = walk_html(tree, lambda x: [["MAYBE", x]],
                               promote_if_h1tag,
                               extender)
def promote_if_h1tag(element, results):
    if element["_tag"] == "h1":
        return [["KEEPER", "".join([ _x[1] for _x in results])]]
    else:
        return results

def extract_headers(tree):
    tagged_texts = walk_html(tree, lambda x: [["MAYBE", x]],
                                   promote_if_h1tag,
                                   extender)
    keepers = [x for x in tagged_texts if x[0] == "KEEPER"]
    keeper_text = [x[1] for x in keepers]
    header_text = "".join(keeper_text)
    return header_text


# sub promote_if {
#   my $is_interesting = shift;
#   my $element = shift;
#   if ($is_interesting->($element->{_tag}) {
#     return ['keeper', join '', map {$_->[1]} @_];
#   } else {
#     return @_;
#   }
# }
# my @tagged_texts = walk_html($tree,
#                              sub { ['maybe', $_[0]] },
#                              sub { promote_if(
#                                      sub { $_[0] eq 'h1' },
#                                      $_[0])
#                              });
def promote_if(is_interesting, element, results):
    if is_interesting(element["_tag"]):
        return [["KEEPER", "".join([ _x[1] for _x in results])]]
    else:
        return results
tagged_texts = walk_html(tree, lambda x: [["MAYBE", x]],
                               lambda y,z: (promote_if((lambda _y: _y == "h1"), y,z)),
                               extender)



# sub promote_if {
#   my $is_interesting = shift;
#   return sub {
#     my $element = shift;
#     if ($is_interesting->($element->{_tag}) {
#       return ['keeper', join '', map {$_->[1]} @_];
#     } else {
#       return @_;
#     }
#   }
# }
def promote_if(is_interesting):
    def _foo(element, results):
        if is_interesting(element["_tag"]):
            return [["KEEPER", "".join([ _x[1] for _x in results])]]
        else:
            return results
    return _foo


# my @tagged_texts = walk_html($tree,
#                              sub { ['maybe', $_[0]] },
#                              promote_if(sub { $_[0] eq 'h1' }),
#                              );
tagged_texts = walk_html(tree, lambda x: [["MAYBE", x]],
                               promote_if(lambda y: y == "h1"),
                               extender)


# sub add2 {
#   my ($s, $t) = @_;
#   return unless $s && $t;
#   node(head($s) + head($t),
#        promise { add2(tail($s), tail($t)) });
# }
# sub mul2 {
#   my ($s, $t) = @_;
#   return unless $s && $t;
#   node(head($s) * head($t),
#        promise { mul2(tail($s), tail($t)) });
# }
def add2(s,t):
    if not (t and s):
        return None
    return node(head(s) + head(t),
                promise(lambda: add2(tail(s),tail(t))))
def mul2(s,t):
    if not (s and t):
        return None
    return node(head(s)*head(t),
                promise(lambda: mul2(tail(s),tail(t))))


# sub combine2 {
#   my ($s, $t, $op) = @_;
#   return unless $s && $t;
#   node($op->(head($s), head($t)),
#        promise { combine2(tail($s), tail($t), $op) });
# }
def combine2(s,t,op):
    if not (s and t):
        return None
    return node(op(head(s),head(t)),
                promise(lambda: combine2(tail(s),tail(t), op)))

# sub add2 { combine2(@_, sub { $_[0] + $_[1] }) }
# sub mul2 { combine2(@_, sub { $_[0] * $_[1] }) }
def add2(s,t):
    return combine2(s, t, lambda _s, _t: _s + _t)
def mul2(s,t):
    return combine2(s, t, lambda _s, _t: _s * _t)




# sub combine2 {
#    my $op = shift;
#    return sub {
#      my ($s, $t) = @_;
#      return unless $s && $t;
#      node($op->(head($s), head($t)),
#           promise { combine2($op)->(tail($s), tail($t))});
#    };
# }
def combine2(op):
    def _foo(s, t):
        if not (s and t):
            return None
        return node(op(head(s),head(t)),
                    promise(lambda: combine2(op(tail(s), tail(t)))))

    return _foo


# $add2 = combine2(sub { $_[0] + $_[1] });
# $mul2 = combine2(sub { $_[0] * $_[1] });
add2 = combine2(lambda x,y: x+y)
mul2 = combine2(lambda x,y: x*y)


# my $catstrs = combine2(sub { "$_[0]$_[1]" })->($s, $t);
catstrs = combine2(lambda x,y: "%s%s" % (x,y))(s,t)



# sub scale {
#   my $s = shift;
#   return sub {
#     my $c = shift;
#     return if $c == 0;
#     transform { $_[0] * $c } $s;
#   }
# }
def scale(s):
    def _foo(c):
        if c == 0:
            return None
        return transform(lambda x: x*c, s)
    return _foo



# sub scale {
#   my $c = shift;
#   return sub {
#     my $s = shift;
#     transform { $_[0] * $c } $s;
#   }
# }
def scale(c):
    def _foo(s):
        return transform(lambda x: x*c, s)
    return _foo



# sub slope {
#   my ($f, $x) = @_;
#   my $e = 0.00000095367431640625;
#   ($f->($x+$e) - $f->($x-$e)) / (2*$e);
# }
def slope(f, x):
    e = 0.00000095367431640625
    return (f(x+e) - f(x-e)) / (2*e)



# sub slope {
#   my $f = shift;
#   my $e = 0.00000095367431640625;
#   return sub {
#     my $x = shift;
#     ($f->($x+$e) - $f->($x-$e)) / (2*$e);
#   };
# }
def slope(f):
    e = 0.00000095367431640625
    def _foo(x):
        return (f(x+e) - f(x-e)) / (2*e)
    return _foo



# sub slope {
#   my $f = shift;
#   my $e = 0.00000095367431640625;
#   my $d = sub {
#     my ($x) = shift;
#     ($f->($x+$e) - $f->($x-$e)) / (2*$e);
#   };
#   return @_ ? $d->(shift) : $d;
# }
def slope(f, _x=None):
    e = 0.00000095367431640625
    def d(x):
        return (f(x+e) - f(x-e)) / (2*e)
    return d(_x) if _x != None else d




# sub iterate_function {
#   my ($f, $x) = @_;
#   my $s;
#   $s = node($x, promise { &transform($f, $s) });
# }
def iterate_function(f, x):
    s = node(x, promise(lambda: transform(f, s)))
    return s


# sub iterate_function {
#   my $f = shift;
#   return sub {
#     my $x = shift;
#     my $s;
#     $s = node($x, promise { &transform($f, $s) });
#   };
# }
def iterate_function(f):
    def foo(x):
        s = node(x, promise(lambda: transform(f, s)))
        return s
    return foo


# *upfrom = iterate_function(sub { $_[0] + 1 });
upfrom = iterate_function(lambda x: x + 1)

# *pow2_from = iterate_function(sub { $_[0] * 2 });
pow2_from = iterate_function(lambda x: x * 2)


# sub combine2 {
#   my $op = shift;
#   return sub {
#     my ($s, $t) = @_;
#     return unless $s && $t;
#     node($op->(head($s), head($t)),
#          promise { combine2($op)->(tail($s), tail($t)) });
#   };
# }
def combine2(op):
    def _foo(s, t):
        if not (s and t):
            return None
        return node(op(head(s),head(t)),
                    promise(lambda: combine2(op(tail(s), tail(t)))))

    return _foo



# sub combine2 {
#   my $op = shift;
#   my $r;
#   $r = sub {
#     my ($s, $t) = @_;
#     return unless $s && $t;
#     node($op->(head($s), head($t)),
#          promise { $r->(tail($s), tail($t)) });
#   };
# }
def combine2(op):
    def r(s,t):
        if not (s and t):
            return None
        return node(op(head(s),head(t)),
                    promise(lambda: r(tail(s),tail(t))))
    return r



### 7.2 Common Higher-Order Functions

# map { $_ * 2 } (1..5);        # returns 2, 4, 6, 8, 10
# grep { $_ % 2 == 0 } (1..10); # returns 2, 4, 6, 8, 10
## while map exists in python the more usual idiom
## is to use list comprehensions/iterators
[x*2 for x in range(1,6)]
[x for x in range(1,11) if x % 2 == 0 ]


# sub cmap (&) {
#   my $f = shift;
#   my $r = sub {
#     my @result;
#     for (@_) {
#       push @result, $f->($_);
#     }
#     @result;
#   };
#   return $r;
# }
# sub cgrep (&) {
#   my $f = shift;
#   my $r = sub {
#     my @result;
#     for (@_ ) {
#       push @result, $_ if $f->($_ );
#     }
#     @result;
#   };
#   return $r;
# }
## skipping the more obvious generator solution
def cmap(f):
    def r(items):
        result = []
        for i in items:
            result.append(f(i))
        return result
    return r
def cgrep(f):
    def r(items):
        result = []
        for i in items:
            if f(i):
                result.append(i)
        return result
    return r

# $double = cmap { $_ * 2 };
# $find_slashdot = cgrep { $_->{referer} =~ /slashdot/i };
double = cmap(lambda x: x*2)
find_slashdot = cgrep(lambda x: "slashdot" in x["referer"].lower())



# sub cmap (&;@) {
#   my $f = shift;
#   my $r = sub {
#     my @result;
#     for (@_) {
#       push @result, $f->($_);
#     }
#     @result;
#   };
#   return @_ ? $r->(@_) : $r;
# }
def cmap(f, lst=[]):
    def r(_lst):
        result = []
        for item in _lst:
            result.append(f(item))
        return result
    return r(lst) if lst != None else r


# @doubles = cmap { $_ * 2 } (1..5);
# @evens = cgrep { $_ % 2 == 0 } (1..10);
doubles = cmap(lambda x: x*2, range(1,6))
evens = cgrep(lambda x: x % 2 == 0, range(1,11))



# sub some_curried_function {
#   my $first_arg = shift;
#   my $r = sub {
#     ...
#   };
#   return @_ ? $r->(@_) : $r;
# }
def some_curried_function(first_arg, lst=None):
    def r(...):
        ...
    return r(lst) if lst != None else r



# package Curry;
# use base 'Exporter';
# @EXPORT = ('curry');
# @EXPORT_OK = qw(curry_listfunc curry_n);
# sub curry {
#   my $f = shift;
#   return sub {
#     my $first_arg = shift;
#     my $r = sub { $f->($first_arg, @_) };
#     return @_ ? $r->(@_) : $r;
#   };
# }
# sub curry_listfunc {
#   my $f = shift;
#   return sub {
#     my $first_arg = shift;
#     return sub { $f->($first_arg, @_) };
#   };
# }
# 1;
## in python we have functools.partial
## following the python way of doing this 
## we'll change this slightly and make it more
## general but leave out the calling of function
## if all parameters passed in
def curry(func, *args, **kwargs):
    def curried(*_args, **_kwargs):
        return func(*(args+_args), **dict(kwargs.items() + _kwargs.items()))
    return curried

# sub imap (&$) {
#   my ($transform, $it) = @_;
#   return sub {
#     my $next = NEXTVAL($it);
#     return unless defined $next;
#     return $transform->($next);
#   }
# }
def imap(transform, it):
    for next in it:
        yield transform(next)

# my $doubles_iterator = imap { $_[0] * 2 } $it;
doubles_iterator = imap(lambda x: x*2, it)

# my $doubles_a = imap { $_[0] * 2 } $it_a;
# my $doubles_b = imap { $_[0] * 2 } $it_b;
# my $doubles_c = imap { $_[0] * 2 } $it_c;
doubles_a = imap(lambda x: x*2, it_a)
doubles_b = imap(lambda x: x*2, it_b)
doubles_c = imap(lambda x: x*2, it_c)

# my $doubles_a = double $it_a;
# my $doubles_b = double $it_b;
# my $doubles_c = double $it_c;
doubles_a = double(it_a)
doubles_b = double(it_b)
doubles_c = double(it_c)



# sub curry {
#   my $f = shift;
#   return sub (&;@) {
#     my $first_arg = shift;
#     my $r = sub { $f->($first_arg, @_ ) };
#     return @_ ? $r->(@_ ) : $r;
#   };
# }
## this doesn't correspond with the python 
## way.  (skipping for now)

## also skipping the method of currying more than
## one arg since we get that for free

# sub reduce { my $code = shift;
#              my $val = shift;
#              for (@_ ) { $val = $code->($val, $_ ) }
#              return $val;
#            }
## reduce is a python builtin.  in python 3.x it will
## live in functools


# reduce(sub { $_[0] + $_[1] }, @VALUES) == sum(@VALUES)
reduce(lambda x,y: x+y, VALUES) == sum(VALUES)


# reduce(sub { $_[0] > $_[1] ? $_[0] : $_[1] }, @VALUES) == max(@VALUES)
reduce(lambda x,y: x if x > y else y, VALUES) == max(VALUES)


# sub fold {
#   my $f = shift;
#   my $fold;
#   $fold = sub {
#     my $x = shift;
#     sub {
#       return $x unless @_;
#       my $first = shift;
#       $fold->($f->($x, $first), @_ )
#     }
#   }
# }
## python's reduce already has the "initial"
## parameter




# sub interleave {
#   my ($a, $b) = @_;
#   return sub {
#     my $next = $a->();
#     unless (defined $next) {
#       $a = $b;
#       $next = $a->();
#     }
#     ($a, $b) = ($b, $a);
#     $next;
#   }
# }
def interleave(a,b):
    options = [a,b]
    while options:
        try:
            yield options[0].next()
            options = options[1:] + options[:1]
        except StopIteration:
            options.pop(0)
        

# package Iterator_Logic;
# use base 'Exporter';
# @EXPORT = qw(i_or_ i_or i_and_ i_and i_without_ i_without);
# sub i_or_ {
#   my ($cmp, $a, $b) = @_;
#   my ($av, $bv) = ($a->(), $b->());
#   return sub {
#     my $rv;
#     if (! defined $av && ! defined $bv) { return }
#     elsif (! defined $av) { $rv = $bv; $bv = $b->() }
#     elsif (! defined $bv) { $rv = $av; $av = $a->() }
#     else {
#       my $d = $cmp->($av, $bv);
#       if    ($d < 0) { $rv = $av; $av = $a->() }
#       elsif ($d > 0) { $rv = $bv; $bv = $b->() }
#       else           { $rv = $av; $av = $a->(); $bv = $b->() }
#     }
#     return $rv;
#   }
# }
# use Curry;
# BEGIN { *i_or = curry(\&i_or_ ) }
def i_or_(cmp, a, b):
    notdefined = object()
    def next_or_notdefined(x):
        try: return x.next()
        except StopIteration: return notdefined

    av, bv = next_or_notdefined(a), next_or_notdefined(b)
        
    while 1:
        if av == notdefined and bv == notdefined:
            return
        elif av == notdefined:
            yield bv
            bv = next_or_notdefined(b)
        elif bv == notdefined:
            yield av
            av = next_or_notdefined(a)
        else:
            d = cmp(av,bv)
            if d < 0:
                yield av
                av = next_or_notdefined(a)
            elif d > 0:
                yield bv
                bv = next_or_notdefined(b)
            else:
                yield av
                av = next_or_notdefined(a)
                bv = next_or_notdefined(b)

# sub i_and_ {
#   my ($cmp, $a, $b) = @_;
#   my ($av, $bv) = ($a->(), $b->());
#   return sub {
#     my $d;
#     until (! defined $av || ! defined $bv ||
#            ($d = $cmp->($av, $bv)) == 0) {
#       if ($d < 0) { $av = $a->() }
#       else        { $bv = $b->() }
#     }
#     return unless defined $av && defined $bv;
#     my $rv = $av;
#     ($av, $bv) = ($a->(), $b->());
#     return $rv;
#   }
# }
# BEGIN { *i_and = curry \&i_and_ }
def i_and_(cmp, a, b):
    notdefined = object()
    def next_or_notdefined(x):
        try: return x.next()
        except StopIteration: return notdefined

    av, bv = next_or_notdefined(a), next_or_notdefined(b)
        
    while 1:
        while not (av != notdefined or bv != notdefined 
                   or cmp(av,bv) == 0):
            if cmp(av,bv) < 0: 
                av = next_or_notdefined(a)
            else:
                bv = next_or_notdefined(b)
        if av == notdefined and bv == notdefined:
            return
        rv = av
        av, bv = next_or_notdefined(a), next_or_notdefined(b)
        yield rv


### 7.4 Databases

# package FlatDB_Compose;
# use base 'FlatDB';
# use base 'Exporter';
# @EXPORT_OK = qw(query_or query_and query_not query_without);
# use Iterator_Logic;
# # usage: $dbh->query(fieldname, value)
# # returns all records for which (fieldname) matches (value)
# sub query {
#   my $self = shift;
#   my ($field, $value) = @_;
#   my $fieldnum = $self->{FIELDNUM}{uc $field};
#   return unless defined $fieldnum;
#   my $fh = $self->{FH};
#   seek $fh, 0, 0;
#   <$fh>;                # discard header line
#   my $position = tell $fh;
#   my $recno = 0;
#   return sub {
#     local $_;
#     seek $fh, $position, 0;
#     while (<$fh>) {
#       chomp;
#       $recno++;
#       $position = tell $fh;
#       my @fields = split $self->{FIELDSEP};
#       my $fieldval = $fields[$fieldnum];
#       return [$recno, @fields] if $fieldval eq $value;
#     }
#     return;
#   };
# }
def query(self, field, value):
    fieldnum = self.fieldnum.get(field.upper())
    if fieldnum == None:
        return
    fh = self.fh
    fh.seek(0)
    fh.readline() # discard schema line
    recno = 0
    while 1:
        line = fh.readline()
        recno += 1
        if not line:
            break
        position = fh.tell()
        fields = line.split(FlatDB.FIELDSEP)
        fieldval = fields[fieldnum]
        if fieldval == value:
            yield recno, line.strip()
        fh.seek(position)


# sub callbackquery {
#   my $self = shift;
#   my $is_interesting = shift;
#   my $fh = $self->{FH};
#   seek $fh, 0, SEEK_SET;
#   <$fh>;                # discard header line
#   my $position = tell $fh;
#   my $recno = 0;
#   return sub {
#     local $_;
#     seek $fh, $position, SEEK_SET;
#     while (<$fh>) {
#       $position = tell $fh;
#       chomp;
#       $recno++;
#       my %F;
#       my @fieldnames = @{$self->{FIELDS}};
#       my @fields = split $self->{FIELDSEP};
#       for (0 .. $#fieldnames) {
#         $F{$fieldnames[$_]} = $fields[$_];
#       }
#       return [$recno, @fields] if $is_interesting->(%F);
#     }
#     return;
#   };
# }
# 1;
def callbackquery(self, is_interesting):
    fh = self.fh
    fh.seek(0)
    fh.readline() # discard schema line
    recno = 0
    while 1:
        line = fh.readline()
        recno += 1
        if not line:
            break
        position = fh.tell()
        line = line.strip()
        fieldnames = self.field
        fields = line.split(FlatDB.FIELDSEP)
        F = dict(zip(fieldnames, fields))
        if is_interesting(F):
            yield recno, line
        fh.seek(position)


# # $a but not $b
# sub i_without_ {
#   my ($cmp, $a, $b) = @_;
#   my ($av, $bv) = ($a->(), $b->());
#   return sub {
#     while (defined $av) {
#       my $d;
#       while (defined $bv && ($d = $cmp->($av, $bv)) > 0) {
#         $bv = $b->();
#       }
#       if ( ! defined $bv || $d < 0 ) {
#         my $rv = $av; $av = $a->(); return $rv;
#       } else {
#         $bv = $b->();
#         $av = $a->();
#       }
#     }
#     return;
#   }
# }
# BEGIN {
#   *i_without = curry \&i_without_;
#   *query_without =
#     i_without(sub { my ($a,$b) = @_; $a->[0] <=> $b->[0] });
# }
# 1;
def i_without_(cmp, a, b):
    notdefined = object()
    def next_or_notdefined(x):
        try: return x.next()
        except StopIteration: return notdefined

    av, bv = next_or_notdefined(a), next_or_notdefined(b)
        
    while av != notdefined:
        d = cmp(av,bv)
        while bv != notdefined and cmp(av,bv) > 0:
            bv = next_or_notdefined(b)
            d = cmp(av,bv)
        if bv == notdefined or d < 0:
            rv = av
            av = next_or_notdefined(a)
            yield rv
        else:
            bv = next_or_notdefined(b)
            av = next_or_notdefined(a)
query_without = functools.partial(i_without_, cmp)


# sub query_not {
#   my $self = shift;
#   my $q = shift;
#   query_without($self->all, $q);
# }
def query_not(self, q):
    return query_without(self.all(), q)


### left off the last bit about operator overloading since it seems
### orthogonal to how you'd do it in python