Riddle 4 Answer

So apparently this is an annual series now…

Anyway, Dan Nugent and Akash were both on the right track, but neither had it exactly right: when a qsql query inside a function is executed, relative names used in it resolve against globals in the current namespace, not the namespace that was in effect when the function was created (i.e. the one returned by running {(get x). 3 0} on the function).

(Note that this actually applies to any type of global (e.g. an atom, a vector, etc.) referenced from a query inside a function, but for convenience, I’ll be writing this post assuming a function is what’s being referenced.)

I will admit to making this a bit of a trick question, as the way I constructed the example was designed around the most common case, which is entirely consistent with Dan’s and Akash’s answers. Here's a snippet showing the full behavior:

% q
KDB+ 3.3 2015.09.02 Copyright (C) 1993-2015 Kx Systems
m32/ 16()core 8192MB adavies aaron-daviess-mac-pro.local 192.168.1.151 NONEXPIRE  

q)f:{x+1}
q)\d .foo
q.foo)g:{select f a from x}
q.foo)\d .bar
q.bar).foo.g([]a:1 2 3)
{select f a from x}
'f
q.foo))\
q.bar)f:{x+2}
q.bar).foo.g([]a:1 2 3)
a
-
3
4
5
q.bar)

Compare to this snippet, where I reference the function from outside the qsql query:

% q
KDB+ 3.3 2015.09.02 Copyright (C) 1993-2015 Kx Systems
m32/ 16()core 8192MB adavies aaron-daviess-mac-pro.local 192.168.1.151 NONEXPIRE  

q)\d .foo
q.foo)f:{x+1}
q.foo)g:{f select a from x}
q.foo)\d .
q).foo.g([]a:1 2 3)
a
-
2
3
4
q)

This can become problematic if you try to create a group of functions in a namespace, some of which reference each other in queries, and then call those functions from outside that namespace. I’ve found two solutions for this, both unfortunately rather inelegant.

1. You can “copy” the function you need from the global space to a local variable by referencing it from outside any qsql queries:

   % q
   KDB+ 3.3 2015.09.02 Copyright (C) 1993-2015 Kx Systems
   m32/ 16()core 8192MB adavies aaron-daviess-mac-pro.local 192.168.1.151 NONEXPIRE  
   
   q)\d .foo
   q.foo)f:{x+1}
   q.foo)g:{f0:f;select f0 a from x}
   q.foo)\d .
   q).foo.g([]a:1 2 3)
   a
   -
   2
   3
   4
   q)

2. You can do the name resolution yourself, by leveraging the get results to automatically reference the correct namespace:

   % q
   KDB+ 3.3 2015.09.02 Copyright (C) 1993-2015 Kx Systems
   m32/ 16()core 8192MB adavies aaron-daviess-mac-pro.local 192.168.1.151 NONEXPIRE  
   
   q)\d .foo
   q.foo)f:{x+1}
   q.foo)g:{select((` sv`,((get .z.s). 3 0))`f)a from x}
   q.foo)\d .
   q).foo.g([]a:1 2 3)
   a
   -
   2
   3
   4
   q)

   This can be encapsulated in a utility function, but note that it must then itself be referenced by an absolute name, or the same problem will apply to it:
   

   % q
   KDB+ 3.3 2015.09.02 Copyright (C) 1993-2015 Kx Systems
   m32/ 16()core 8192MB adavies aaron-daviess-mac-pro.local 192.168.1.151 NONEXPIRE  
   
   q)\d .util
   q.util)r:{(` sv`,((get x). 3 0))y}
   q.util)\d .foo
   q.foo)f:{x+1}
   q.foo)g:{select((.util.r .z.s)`f)a from x}
   q.foo)\d .
   q).foo.g([]a:1 2 3)
   a
   -
   2
   3
   4
   q)

Riddle 3 Answer

And it looks like I did it again—Riddle 3 has been lacking an official answer for almost a year!
Ciaran Gorman’s answer was correct—strings with leading and/or trailing space are exactly what I was thinking of, and the serialization technique he showed is how to deal with them.
The following function should work as a general solution:

{0x01,($[.z.o like"s*";reverse;::]0x0 vs"i"$10+count x),0x000000f5,("x"$x),0x00}

Riddle 2 Answer

This is somewhat esoteric, and I wouldn’t be surprised if very few people had any idea what I was even asking. I discovered this largely by chance, while fiddling around with function bytecode, though I think it could be deduced from observation without that.

So, the answer:

:: is usually taught as a sui generis operator, called “global amend”, which has the specific behavior (when used as a verb inside a function) of setting a global variable (instead of the local one that : would set in the same place). No connection is typically drawn between it and any other operator (other than :).

However, I’m pretty sure this is inaccurate. While obviously I don’t know for certain, I strongly suspect that there is no code anywhere in the q binary saying that :: is defined as “global amend”. Rather, it is a specific case of the dyadic “f:” pattern, where f is some dyadic function—e.g. dyadic +:, -:, *:, etc.

These all have the same behavior—x f:y is defined as x:x f y.

Additionally, when used inside functions on variables that have not been identified by the compiler as locals, they modify (and if necessary, create), global variables.

q){a:1;a+:1;a}[]
2
q)a
'a
q){a+:1}[]
q)a
1
q)

It follows that if f is :, then the operation involved is assignment, and so x gets y assigned to it, as a global variable if not identified as a local variable.

In fact, this can be seen in the same way:

q){a:1;a::2;a}[]
2
q)a
'a
q){a::2}[]
q)a
2
q)

Thus arises “global amend”.

If anything, the “create view” sense of :: must be the special case, as ordinarily, dyadic f: verbs behave identically inside and outside functions.

Riddle 1 Answer

I appear to have left Riddle 1 sitting out there without an official answer for almost seven months now. Sorry about that.

The answer given by Peter Byrne was valid, and essentially the one I was thinking of: while his example dealt with the untyped empty list (), I had the typed empty list `boolean$() in mind.

The insight here is that any and all are forms of min and max; and that min x,y, the min of the concatenation of two lists, is equal to min(min x;min y), the min of their separate mins (and mutatis mutandis for max). For this to work consistently for empty lists, the min of an empty list must be the maximum possible value for that data type (and mutatis mutandis for max).

Riddle 0 Answer

As several people guessed in the comments, the answer I had in mind was what I think of as “compressed” matrices—general lists where some entries are lists of the same length and others are atoms.

q)(1f;`a`b)
1f
`a`b
q)flip flip(1f;`a`b)
1 1
a b
q)

This is, as far as I can tell, directly related to atomic extension (x f'y behaves identically for vector/vector, atom/vector, and vector/atom) and similar concepts, such as the ability to use an atom for a constant column in a table literal (([]x:1 2 3;y:4)).

(My original intention was to post these more or less weekly; hopefully I’ll be able to stick a bit closer to that in future.)