ECMAScript spread param rules causing loss of type-safety (and slower compile time?)

Mohamed Hegazy (@mhegazy)

I think we're hitting a similar situation to #1336 with RxJS 5, which is causing some crazy workarounds for some minimal type-safety, which in turn is causing slower build times.

Ideally what we'd have is this:

// Observable.prototype.concat
concat<R>(...observables: ObservableInput<any>[], scheduler?: Scheduler): Observable<R>

But since that's illegal, we're forced to do this:

concat<R>(...args: Array<ObservableInput<any> | Scheduler>): Observable<R>

... that clearly doesn't give us what we want for type safety, so we end up defining a crazy list of common type signatures like:

/* tslint:disable:max-line-length */
export interface ConcatSignature<T> {
  (scheduler?: Scheduler): Observable<T>;
  <T2>(v2: ObservableInput<T2>, scheduler?: Scheduler): Observable<T | T2>;
  <T2, T3>(v2: ObservableInput<T2>, v3: ObservableInput<T3>, scheduler?: Scheduler): Observable<T | T2 | T3>;
  <T2, T3, T4>(v2: ObservableInput<T2>, v3: ObservableInput<T3>, v4: ObservableInput<T4>, scheduler?: Scheduler): Observable<T | T2 | T3 | T4>;
  <T2, T3, T4, T5>(v2: ObservableInput<T2>, v3: ObservableInput<T3>, v4: ObservableInput<T4>, v5: ObservableInput<T5>, scheduler?: Scheduler): Observable<T | T2 | T3 | T4 | T5>;
  <T2, T3, T4, T5, T6>(v2: ObservableInput<T2>, v3: ObservableInput<T3>, v4: ObservableInput<T4>, v5: ObservableInput<T5>, v6: ObservableInput<T6>, scheduler?: Scheduler): Observable<T | T2 | T3 | T4 | T5 | T6>;
  (...observables: Array<ObservableInput<T> | Scheduler>): Observable<T>;
  <R>(...observables: Array<ObservableInput<any> | Scheduler>): Observable<R>;
}
/* tslint:enable:max-line-length */

I believe this is directly related to compilation speed issues that Mohamed Hegazy (@mhegazy) wanted to discuss with me.

cc/ OJ Kwon (@kwonoj) David Driscoll (@david-driscoll) Jay Phelps (@jayphelps)

Jay Phelps (@jayphelps)

Okay, I misunderstood what Jay Phelps (@jayphelps) was saying... but there is still one problem with his proposal, which is that the implementation signature could still be matched, which wouldn't keep the function type-safe.

eg, this:

function concat(...observables: Array<Observable>, scheduler: Scheduler);
function concat(...args: Array<Observable | Scheduler>) {
  console.log(args.length);
  // 3
}

concat(new Observable, new Observable, new Scheduler);

Also matches this:

concat(new Scheduler, new Scheduler, new Observable);

(discussed this with Jay in person a minute ago)

The implementation signature isn't visible to callers (for exactly this reason!) -- from the "outside", you only see the (...observables: Array<Observable>, scheduler: Scheduler) signature

Ryan Cavanaugh (@RyanCavanaugh)

Ryan Cavanaugh (@RyanCavanaugh) seeing that too. Emulating this behavior, I see it suggest only that signature (a pseudo variant of it, that is cause obv this syntax isn't supported). But it still let me provide the arguments in the wrong order, as ben noted above.

So if this was added as a valid overload signature, it would make things "type-safe" for people consuming compiled .js files and their .d.ts counterparts, I guess. Because TypeScript seems to allow matching the implementation signature if you're using the .ts file directly. ... I guess you'd need a way to say "this signature is for implementation only and we don't want you using it like that". Which seems odd.

Because TypeScript seems to allow matching the implementation signature if you're using the .ts file directly

This isn't true.

function f(x: number): void;
function f(x: string): void;
function f(x: any): void { }

// Errors; this cannot see the x: any overload
f({q: 2});

Mohamed Hegazy (@mhegazy)

This isn't true.

Ah, thanks for the clarification, Mohamed Hegazy (@mhegazy)

From the OP it seems that this proposal is not sufficient, and that variadic types are also required. this is based on the current list of signatures above you have opted to use generic type parameters for every input and accumulated in the output Observable<T | T2 | T3 | T4 | T5 | T6>. having said that, variadic types are tracked by #5453, and whereas a leading rest signature might be a small change, variadic generic types are a whole different beast. so it seems that just leading rest args might make the last overload more type safe but does not make the overload list any shorter or simpler.

Mohamed Hegazy (@mhegazy)

Mohamed Hegazy (@mhegazy) Yep, you're right, with one clarification. Ben will have to confirm but my discussions with him led me to believe that most operators (including concat) actually do not need variadic generic types. Only those that provide a callback like zip. We believe that a majority of operators only need a single generic for return value. The extra generics that are used right now in concat et al are extraneous as fas we we can tell.

function concat<R>(...observables: Array<ObservableInput<any>>, scheduler?: Scheduler): Observable<R>;
function concat<R>(...args: Array<ObservableInput<any> | Scheduler>): Observable<R> {
  console.log(args.length);
  // 3
}

So while RxJS will need variadic generics to make all operators non-final rest params in overloads gets us ~80% closer.

Anyone feel free to correct us if you believe otherwise. You guys know the compiler limits far better than we do of course!

The existence of the uninferrable <R> type parameter is somewhat confusing. What's it there for?

Ryan Cavanaugh (@RyanCavanaugh)

Ryan Cavanaugh (@RyanCavanaugh) I think so users can type the returned Observable explicitly if they so choose? I'm unsure of the original intent there, if I'm honest.

Mohamed Hegazy (@mhegazy)

From the OP it seems that this proposal is not sufficient, and that variadic types are also required.

Mohamed Hegazy (@mhegazy), you're exactly right. We' have issues with the zip, combineLatest and withLatestFrom operators, specifically, because they're used like this (for example):

const obsA = new Observable<A>();
const obsB = new Observable<B>();
const obsC = new Observable<C>();

obsA.withLatestFrom(obsB, obsC, (a: A, b: B, c: C) => new ReturnType(a, b, c)); // Observable<ReturnType> expected

So you'd almost need a type signature like this (probably poorly designed) psuedocode:

withLatestFrom<...T, R>(...observables: ...Observable<T>, (...args: ...T) => R): Observable<R>

Mohamed Hegazy (@mhegazy)

... however, Mohamed Hegazy (@mhegazy), for concat and merge and others, just the proposed solution should work, since there's really no good reason we need to be passing T1, T2, etc with those. They could (and probably should) just be Observable<any> arguments.

Jay Phelps (@jayphelps)

Further, talking with Jay Phelps (@jayphelps), we don't really need the return type on concat to be anything in particular, and could have it return Observable<any> I think.

const result: Observable<number> = obs1.concat<number>(obs2);

is the same as

const result: Observable<number> = obs1.concat(obs2);

... if concat returned Observable<any>.

cc/ David Driscoll (@david-driscoll) OJ Kwon (@kwonoj).

the generic types on concat, merge are useful because they let us resolve something like...

let obs1: Observable<number>;
let obs2: Observable<number>;
const result: Observable<number> = obs1.concat(obs2);

to something like....

let obs1: Observable<number>;
let obs2: Observable<number>;
const result = obs1.concat(obs2);

The less manual inference I as the consumer has to do, the easier my life is. In addition if you make a small incompatible change at the top of a set of operations, it cascades gracefully down letting you see how badly you failed.

let obs1: Observable<{item: number}>;
let obs2: Observable<string>;
const result = obs1
    .concat(obs2)
    .map(x => x.item * 2) // would fail saying `x` doesn't have a property of `item`

I'm fine with ignoring variadics for the moment to deal with this specific scenario (rest args), but variadics will be useful going forward beyond just combineLatest and zip.

Now to note, for concat/merge we could drop the requirement of variadics and just assume we're getting a named type T, and then returning Observable<T>. That covers most situations, but then some operations, where you want to have a stream of Apples or Oranges would have to be explicitly typed.

This pattern seems to be relatively rare and would be quite complex to handle during overload resolution, which makes tons of assumptions about parameter ordering. If we see other libraries putting things at the end we can reconsider (i.e. if you encounter other functions with this pattern, please leave a comment with a link).

Ryan Cavanaugh (@RyanCavanaugh)

Ryan Cavanaugh (@RyanCavanaugh) understandable.

It's definitely not a super common problem. I imagine it's usually APIs dealing with a callback, since they're typically the last arg but even then, not something I see often.

This pattern seems to be relatively rare

It seems like Angular 1 had this pattern everywhere for their DI, or at least something very similar.

Either way, I'm pleased that you at least considered our needs. 👍

Blesh

Since this is relevant to my current endeavors I'm going to comment here in the case it gets revisited.

In attempting to convert one of our webpack core libraries, I too, came across the same issue when trying to implement the following: https://github.com/webpack/tapable#applypluginsasyncseries

This is the current JavaScript implementation:

Tapable.prototype.applyPluginsAsyncSeries = Tapable.prototype.applyPluginsAsync = function applyPluginsAsync(name) {
    var args = Array.prototype.slice.call(arguments, 1);
    var callback = args.pop();
    if(!this._plugins[name] || this._plugins[name].length === 0) return callback();
    var plugins = this._plugins[name];
    var i = 0;
    var _this = this;
    args.push(copyProperties(callback, function next(err) {
        if(err) return callback(err);
        i++;
        if(i >= plugins.length) {
            return callback();
        }
        plugins[i].apply(_this, args);
    }));
    plugins[0].apply(this, args);
};

And the documentation for that function specifying the following:

applyPluginsAsyncSeries

applyPluginsAsyncSeries(
    name: string,
    args: any...,
    callback: (err: Error, result: any) -> void
)

Asynchronously applies all registered handers for name. The hander functions are called with all args and a callback function with the signature (err: Error) -> void. The handers are called in series, one at a time. After all handlers are applied, callback is called. If any handler passes a value for err, the callback is called with this error and no more handlers are called.

I will take the same approach that Blesh et al are taking for RxJS. Only problem is that the number of arguments can reach up to 50+ (because in webpack core, almost every bit of functionality uses this plugin system).

Thanks for tracking this!

UPDATE:

50+ (because in webpack core, almost every bit of functionality uses this plugin system).

I mispoke on this, the arguments list can be pretty large still but plugins registration happens at a different point etc.

Navigation Menu

Repository navigation

ECMAScript spread param rules causing loss of type-safety (and slower compile time?) #7347

applyPluginsAsyncSeries

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

ECMAScript spread param rules causing loss of type-safety (and slower compile time?) #7347

Description

Activity

benlesh commented on Mar 4, 2016

RyanCavanaugh commented on Mar 4, 2016

jayphelps commented on Mar 4, 2016

benlesh commented on Mar 4, 2016

RyanCavanaugh commented on Mar 7, 2016

benlesh commented on Mar 7, 2016

mhegazy commented on Mar 7, 2016

jayphelps commented on Mar 8, 2016

RyanCavanaugh commented on Mar 8, 2016

benlesh commented on Mar 8, 2016

benlesh commented on Mar 8, 2016

benlesh commented on Mar 8, 2016

benlesh commented on Mar 8, 2016

david-driscoll commented on Mar 8, 2016

david-driscoll commented on Mar 8, 2016

RyanCavanaugh commented on Mar 15, 2016

jayphelps commented on Mar 15, 2016

benlesh commented on Mar 21, 2016

benlesh commented on Mar 21, 2016

TheLarkInn commented on Sep 21, 2016

applyPluginsAsyncSeries

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions