Friday, May 20, 2016

Riddle 4: What’s Going On Here?

What’s happening in this snippet, and why is it interesting?
% q
KDB+ 3.3 2016.03.14 Copyright (C) 1993-2016 Kx Systems
m32/ 2()core 2048MB adavies air.local 10.37.129.2 NONEXPIRE

q)f:{x+1}
q)\d .foo
q.foo)g:{select f a from x}
q.foo)\d .
q).foo.g([]a:1 2 3)
a
-
2
3
4
q)

Labels: ,

Riddle 3 Answer

And it looks like I did it again—Riddle 3 has been lacking an official answer for almost a year!
Ciaran Gorman’s answer was correct—strings with leading and/or trailing space are exactly what I was thinking of, and the serialization technique he showed is how to deal with them.
The following function should work as a general solution:
{0x01,($[.z.o like"s*";reverse;::]0x0 vs"i"$10+count x),0x000000f5,("x"$x),0x00}

Labels: , ,

Hello from Montauk!

I’m at KxCon 2016 in Montauk, and having lots of fun so far. Look me up if you’re here!

Thursday, May 19, 2016

github.com/adavies42/qist

q-ist is now on GitHub! I’ve started a new repository, https://github.com/adavies42/qist. My old code from contrib is now available there, but more importantly, I received permission from work to release a collection of utilities and example code from my personal library. This includes a much fuller-featured version of my wtf function, my awq tool for using q as a text filter, dozens of miscellaneous utility functions, and more. I don’t have a README for the whole repository written yet, so to get you started, most of the interesting stuff is in lib, except for awq, which is in bin. Have fun exploring the repository, and feel free to comment or email me with any questions about the code.

Friday, October 16, 2015

A Combinatoric Combinator

I wrote this while I was playing around with using q on a hobby project, and I thought I’d share it in case anyone else might find it useful.

It takes a number k, a function f, and a list or dictionary y of count n, and runs f once for each of the k-combinations of y. The result is returned as a dictionary with the function outputs as its values and its keys determined by the type of y: if y is a list, its keys are the subsets of y that produced the outputs; if y is a dictionary, its keys are the subsets of the keys of y that index the subsets of y that produced the outputs.

eachc:{
    c:(where reverse 0b vs)each c@:where((first x)=sum 0b vs)each c:til"j"$2 xexp count y;
    (last x)peach$[99h=type y;(key each y)!y:y{((key x)y)#x}/:c;y!y:y@/:c]}

Examples:

q)eachc[(3;sum)]til 5
0 1 2| 3
0 1 3| 4
0 2 3| 5
1 2 3| 6
0 1 4| 5
0 2 4| 6
1 2 4| 7
0 3 4| 7
1 3 4| 8
2 3 4| 9
q)eachc[(3;sum)]`a`b`c`d`e!til 5
a b c| 3
a b d| 4
a c d| 5
b c d| 6
a b e| 5
a c e| 6
b c e| 7
a d e| 7
b d e| 8
c d e| 9
q)

Notes and caveats:

On little-endian machines (i.e. Sparc), the reverse will probably need to be removed.

The size of the result set gets very big very quickly—an n of thirty is probably infeasible for most machines.

I’ve written it to execute f on the combinations with peach, rather than each; this may or may not be appropriate, depending on the nature of any given f and y.

Monday, June 8, 2015

Riddle 3: not x~string`$x

Here’s an easy one:

When will {(10h=type x)&not x~string`$x} return true (1b)?

Slightly harder:

In cases where this is problematic, what can be done about it?

Labels: ,

Friday, April 24, 2015

Name! That! Function! (Pivot Table Edition)

What time is it, kids? That’s right, it’s time to play Name! That! Function!

Seriously though, I have a small, useful (IMAO) function I’ve been entering freehand in the console for something like two years now.

Normally, I’d put it in my personal library, but there’s a problem—I can’t think of a good name for it.

Here’s the function: {((union)over key each x)#/:x}.

And here it is in context, showing what it’s good for:

q)t:([]id:1 1 2 2 3;k:`a`b`b`a`b;v:1 2 3 4 5)
q)t
id k v
------
1  a 1
1  b 2
2  b 3
2  a 4
3  b 5
q)exec k!v by id:id from t
id|         
--| --------
1 | `a`b!1 2
2 | `b`a!3 4
3 | (,`b)!,5
q){((union)over key each x)#/:x}exec k!v by id:id from t
id| a b
--| ---
1 | 1 2
2 | 4 3
3 |   5
q)

So, there it is—an easy way to fix up ad-hoc pivots1 when your data doesn’t have all keys present (and in the same order) on all ids.

Anyone have any ideas what to call it?

  1. Note that for production pivots, particularly if they involve significant amounts of data, you should be using an optimized pivot function.