Repository navigation
local and global versions of .I, .N #1206
Description
It's always been a bit confusing to me that .I is "global" in the sense that it doesn't change with by, while .N is "local" in the sense that it does.
I understand (some of) the advantages of this arrangement, but I think there are ample situations for using a local .I (see, e.g. 1, 2, 3, etc.) or a global .N (e.g., 1).
I'm not sure how easy this is to build into the source code, but having .i and .n be "local" while .I and .N are "global" seems like an intuitive alternative. On the other hand, it could be painful to switch the behavior of .N given that it's so ubiquitous in data.table code.
Throwing a hat in the ring for .SD and .sd as well, since I've been tempted a few times to try .SD with the intention of getting the full table within by, specifically here.
Activity
franknarf1 commentedon Jul 3, 2015
I agree. It'd be quite a break from backward compatibility, but that notation would be useful and a lot more intuitive.
jangorecki commentedon Jul 3, 2015
Quite a big chance. Not sure if the performance gain are big enough to use
.iinstead of current1:.N, anybody measured it? 2.0.0 release is going to have some breaking changes so this could be the place to release such change.arunsrinivasan commentedon Jul 7, 2015
Like the idea very much, but not sure if it's possible at this stage, as it'd break a lot of code.. Where were you when this was implemented first :P?
Marked as FR for now.
MichaelChirico commentedon Jul 7, 2015
In my
Rswaddling blankets, I suppose ;)I understand
.N->.nis a big push, but I rarely need that..i, however, shouldn't break any code and I would use it all the time!arunsrinivasan commentedon Jul 7, 2015
Right. But
seq_len(.N)is.iwhat you're looking for.. Is that not okay? I ask because I find the intent quite clear and understandable..iand.Ican get quite confusing quickly. If it's really necessary, then maybe.seqN? Not sure. I'm always on alert when we've to add more symbols :-).MichaelChirico commentedon Jul 7, 2015
It feels pretty natural to me, and writing, e.g.
var[.i<5]is certainly more intuitive (and cleaner!) thanvar[seq_len(.N)<5](or evenvar[.seqN<5]), but maybe that's just me--FWIW I have more of a math than a programming background, which may be why it's easy for me to compartmentalizing capital vs. lowercase symbols.I understand (and appreciate!) the aversion to over-loading
data.tablewith arcane symbols, but<opinion>I think that anyone that can handle.Iand.Ncan conquer.iquickly, given the tight relationship to.I.<\opinion>.Just one more parallel to draw--
var[.N]is redundant withvar[length(var)], but.lenwas eschewed for the (I think) clearly superior.N;.endalso would have worked but would seem more obtuse in other contexts (e.g.var:=.end).Food for thought! Thanks for the consideration.
franknarf1 commentedon Jul 7, 2015
Or...
.GRPI?Personally, I think it would be easier for new users if the shortcuts were revamped to not only include this extra one but also to be consistent in some sense, like
.I&.N/.GRPI&.GRPNor.I&.N/.GI&.GN(also making.Gan alias or replacement for.GRP) or.DTI&.DTN/.I&.NBreaking compatibility like this isn't so great, and I'd settle for
.ior.GRPIor.GIor even.seqN(though it strikes me as too R-ish) alone. I'd use that shortcut all the time.MichaelChirico commentedon Jul 10, 2015
I've added an example to the main post of when my instinct was to use
.nand.Nbut needed to usenrow(dt)insteadfranknarf1 commentedon Apr 25, 2017
Maybe related: it might be nice to have
.NGRPfor the number of groups. E.g., here it could use the conditionif (.GRP != .NGRP)instead ofif(.I[.N] < nrow(DT)). http://stackoverflow.com/a/43615843/This would also be nice for easily tracking progress by throwing a
print(.GRP/.NGRP)intoj.st-pasha commentedon Jan 31, 2018
How about the following scheme
.I(if no groups).I.N(if no groups).Nx.IDTx.NDTi.Iidata.table when joiningi.Nidata.table when joining.SD.SD.I.SD.I.N.SD.N.BY.BY.GRP.BY.I.BY.NMichaelChirico commentedon Jan 31, 2018
Not sure when symbol overload kicks in... certainly most seem intuitive (though I admit I don't immediately get
.BY.I/.BY.N. Why not.GRP.Iand.GRP.N?And why wouldn't
.Nalways be computable? Unless there's some plan for distributingdata.table?The primary concern remains the introduction of code-breaking behavior.
st-pasha commentedon Jan 31, 2018
The idea is that
?.Iis always an index within somedata.table, where?explicitly states which one (and empty?means the data.table which is being constructed, and hence having no name yet). Similarly,?.Nalways denotes the number of rows inN.The symbols
.BY.Iand.BY.Nrefer to data.table.BY, which is the currently existing symbol and it denotes the data.table of all unique group-by keys. On the contrary,.GRPcurrently means the "group counter", so.GRP.I/.GRP.Nwould require change in the meaning of.GRP.I was trying to make a suggestion that is least breaking and most logically consistent. As it stands, it only changes the meaning of
.Iand.N, and only within the group-by context..Nmay not be computable ifjexpression returns a data.table with unpredictable number of rows. So if you have an expression likeDT[, {if(.N>5) .SD else data.table()} ]then it is impossible to know how many rows there will be in the resulting data.table (which is the new meaning of.N) until you actually construct that data.table.MichaelChirico commentedon Mar 13, 2018
Adding this potentially confusing syntax to this issue (not sure if worth fixing):
[-]Feature Request: local and global versions of `.I`, `.N`[/-][+]local and global versions of `.I`, `.N`[/+]daynefiler commentedon Nov 10, 2023
Just wanted to ping this as a great feature request -- I would very regularly use a version of
.Iand second the proposal for.GRPI.