Friday, December 30, 2016

Riddle 4 Answer

So apparently this is an annual series now…

Anyway, Dan Nugent and Akash were both on the right track, but neither had it exactly right: when a qsql query inside a function is executed, relative names used in it resolve against globals in the current namespace, not the namespace that was in effect when the function was created (i.e. the one returned by running {(get x). 3 0} on the function).

(Note that this actually applies to any type of global (e.g. an atom, a vector, etc.) referenced from a query inside a function, but for convenience, I’ll be writing this post assuming a function is what’s being referenced.)

I will admit to making this a bit of a trick question, as the way I constructed the example was designed around the most common case, which is entirely consistent with Dan’s and Akash’s answers. Here's a snippet showing the full behavior:
% q
KDB+ 3.3 2015.09.02 Copyright (C) 1993-2015 Kx Systems
m32/ 16()core 8192MB adavies aaron-daviess-mac-pro.local 192.168.1.151 NONEXPIRE  

q)f:{x+1}
q)\d .foo
q.foo)g:{select f a from x}
q.foo)\d .bar
q.bar).foo.g([]a:1 2 3)
{select f a from x}
'f
q.foo))\
q.bar)f:{x+2}
q.bar).foo.g([]a:1 2 3)
a
-
3
4
5
q.bar)
Compare to this snippet, where I reference the function from outside the qsql query:
% q
KDB+ 3.3 2015.09.02 Copyright (C) 1993-2015 Kx Systems
m32/ 16()core 8192MB adavies aaron-daviess-mac-pro.local 192.168.1.151 NONEXPIRE  

q)\d .foo
q.foo)f:{x+1}
q.foo)g:{f select a from x}
q.foo)\d .
q).foo.g([]a:1 2 3)
a
-
2
3
4
q)
This can become problematic if you try to create a group of functions in a namespace, some of which reference each other in queries, and then call those functions from outside that namespace. I’ve found two solutions for this, both unfortunately rather inelegant.
  1. You can “copy” the function you need from the global space to a local variable by referencing it from outside any qsql queries:
    % q
    KDB+ 3.3 2015.09.02 Copyright (C) 1993-2015 Kx Systems
    m32/ 16()core 8192MB adavies aaron-daviess-mac-pro.local 192.168.1.151 NONEXPIRE  
    
    q)\d .foo
    q.foo)f:{x+1}
    q.foo)g:{f0:f;select f0 a from x}
    q.foo)\d .
    q).foo.g([]a:1 2 3)
    a
    -
    2
    3
    4
    q)
  2. You can do the name resolution yourself, by leveraging the get results to automatically reference the correct namespace:
    % q
    KDB+ 3.3 2015.09.02 Copyright (C) 1993-2015 Kx Systems
    m32/ 16()core 8192MB adavies aaron-daviess-mac-pro.local 192.168.1.151 NONEXPIRE  
    
    q)\d .foo
    q.foo)f:{x+1}
    q.foo)g:{select((` sv`,((get .z.s). 3 0))`f)a from x}
    q.foo)\d .
    q).foo.g([]a:1 2 3)
    a
    -
    2
    3
    4
    q)
    This can be encapsulated in a utility function, but note that it must then itself be referenced by an absolute name, or the same problem will apply to it:
    % q
    KDB+ 3.3 2015.09.02 Copyright (C) 1993-2015 Kx Systems
    m32/ 16()core 8192MB adavies aaron-daviess-mac-pro.local 192.168.1.151 NONEXPIRE  
    
    q)\d .util
    q.util)r:{(` sv`,((get x). 3 0))y}
    q.util)\d .foo
    q.foo)f:{x+1}
    q.foo)g:{select((.util.r .z.s)`f)a from x}
    q.foo)\d .
    q).foo.g([]a:1 2 3)
    a
    -
    2
    3
    4
    q)

Labels: , ,

Friday, May 20, 2016

Riddle 4: What’s Going On Here?

What’s happening in this snippet, and why is it interesting?
% q
KDB+ 3.3 2016.03.14 Copyright (C) 1993-2016 Kx Systems
m32/ 2()core 2048MB adavies air.local 10.37.129.2 NONEXPIRE

q)f:{x+1}
q)\d .foo
q.foo)g:{select f a from x}
q.foo)\d .
q).foo.g([]a:1 2 3)
a
-
2
3
4
q)

Labels: ,

Riddle 3 Answer

And it looks like I did it again—Riddle 3 has been lacking an official answer for almost a year!
Ciaran Gorman’s answer was correct—strings with leading and/or trailing space are exactly what I was thinking of, and the serialization technique he showed is how to deal with them.
The following function should work as a general solution:
{0x01,($[.z.o like"s*";reverse;::]0x0 vs"i"$10+count x),0x000000f5,("x"$x),0x00}

Labels: , ,

Hello from Montauk!

I’m at KxCon 2016 in Montauk, and having lots of fun so far. Look me up if you’re here!

Labels:

Thursday, May 19, 2016

github.com/adavies42/qist

q-ist is now on GitHub! I’ve started a new repository, https://github.com/adavies42/qist. My old code from contrib is now available there, but more importantly, I received permission from work to release a collection of utilities and example code from my personal library. This includes a much fuller-featured version of my wtf function, my awq tool for using q as a text filter, dozens of miscellaneous utility functions, and more. I don’t have a README for the whole repository written yet, so to get you started, most of the interesting stuff is in lib, except for awq, which is in bin. Have fun exploring the repository, and feel free to comment or email me with any questions about the code.

Labels:

Friday, October 16, 2015

A Combinatoric Combinator

I wrote this while I was playing around with using q on a hobby project, and I thought I’d share it in case anyone else might find it useful.
It takes a number k, a function f, and a list or dictionary y of count n, and runs f once for each of the k-combinations of y. The result is returned as a dictionary with the function outputs as its values and its keys determined by the type of y: if y is a list, its keys are the subsets of y that produced the outputs; if y is a dictionary, its keys are the subsets of the keys of y that index the subsets of y that produced the outputs.

eachc:{
    c:(where reverse 0b vs)each c@:where((first x)=sum 0b vs)each c:til"j"$2 xexp count y;
    (last x)peach$[99h=type y;(key each y)!y:y{((key x)y)#x}/:c;y!y:y@/:c]}

Examples:
q)eachc[(3;sum)]til 5
0 1 2| 3
0 1 3| 4
0 2 3| 5
1 2 3| 6
0 1 4| 5
0 2 4| 6
1 2 4| 7
0 3 4| 7
1 3 4| 8
2 3 4| 9
q)eachc[(3;sum)]`a`b`c`d`e!til 5
a b c| 3
a b d| 4
a c d| 5
b c d| 6
a b e| 5
a c e| 6
b c e| 7
a d e| 7
b d e| 8
c d e| 9
q)
Notes and caveats:
On little-endian machines (i.e. Sparc), the reverse will probably need to be removed.
The size of the result set gets very big very quickly—an n of thirty is probably infeasible for most machines.
I’ve written it to execute f on the combinations with peach, rather than each; this may or may not be appropriate, depending on the nature of any given f and y.

Labels:

Monday, June 8, 2015

Riddle 3: not x~string`$x

Here’s an easy one:

When will {(10h=type x)&not x~string`$x} return true (1b)?

Slightly harder:

In cases where this is problematic, what can be done about it?

Labels: ,