Wednesday, October 9, 2013

Riddle 1 Answer

I appear to have left Riddle 1 sitting out there without an official answer for almost seven months now. Sorry about that.

The answer given by Peter Byrne was valid, and essentially the one I was thinking of: while his example dealt with the untyped empty list (), I had the typed empty list `boolean$() in mind.

The insight here is that any and all are forms of min and max; and that min x,y, the min of the concatenation of two lists, is equal to min(min x;min y), the min of their separate mins (and mutatis mutandis for max). For this to work consistently for empty lists, the min of an empty list must be the maximum possible value for that data type (and mutatis mutandis for max).

Labels: ,

Quick Tip: Intra-Statement Breakpoints

A statement referencing a non-existent variable is a common way to add a breakpoint to a q function.
q)f:{x:x+1;break;x+2}
While this is handy, it only lets you break between statements. A simple extension lets you break within statements, inspecting values at arbitrary points in the code, and continuing with execution once you’re done:
q)f:{x:x+1;{break;x}x+2}
Now, the break will occur after the portion of the statement to the right of the break function has executed, and x within the break function will have the value returned by that code. Since the break statement itself has no effects, and x contains the value returned by the code executed so far, typing : to continue will allow the rest of the break function to execute, returning x leftwards and allowing the rest of the statement to execute as if nothing had interrupted it.

Note that this is entirely legal inside qsql queries; I’ve often found it of particular utility there, since you can’t create new local variables inside a query. Unfortunately no specific examples come to mind at the moment; I’ll try to post one later to make it clearer how this technique works.

Labels: ,

Thursday, May 30, 2013

Parallel xasc

Sorting a table can be divided into two parts: determining the new order for the rows, and applying that ordering to the columns. While the former can’t be parallelized in q, the latter can. I don’t have any hard numbers handy at the moment, but with large tables and under the right conditions, I’ve seen noticeable speedups.

Note, BTW, that you can’t (and shouldn’t) write to disk from inside a peach, so this is only applicable to an ordinary in-memory table sort, not the on-disk variety (`c xasc`:t).

q)pxasc :{(count keys y)!flip{y x}[ iasc(raze x)#0!y]peach flip 0!y}
q)pxdesc:{(count keys y)!flip{y x}[idesc(raze x)#0!y]peach flip 0!y}

Labels:

Monday, March 11, 2013

My kdb+ User Meeting Presentation

Here are the slides from the presentation I gave today at the kdb+ user meeting at BAML, in keynote, pdf, and powerpoint formats:
UPDATE: and here's the code as a loadable .q file:

Labels: ,

Riddle 1: (all x)and not any x


When will {(all x)and not any x} return true (1b)?

Labels:

Tuesday, December 11, 2012

Riddle 0 Answer

As several people guessed in the comments, the answer I had in mind was what I think of as “compressed” matrices—general lists where some entries are lists of the same length and others are atoms.
q)(1f;`a`b)
1f
`a`b
q)flip flip(1f;`a`b)
1 1
a b
q)
This is, as far as I can tell, directly related to atomic extension (x f'y behaves identically for vector/vector, atom/vector, and vector/atom) and similar concepts, such as the ability to use an atom for a constant column in a table literal (([]x:1 2 3;y:4)).

(My original intention was to post these more or less weekly; hopefully I’ll be able to stick a bit closer to that in future.)

Labels: ,

Sunday, November 4, 2012

Riddle 0: not x~flip flip x

I’m going to start occasionally posting q riddles; this is the first.

When will {x~flip flip x} return false (0b)?

Labels: