镜像站点 · 本页由第三方 GitHub 只读镜像提供，非 GitHub 官方站点，不接受任何登录或凭据输入。前往 github.com
Skip to content

[Discussion]: Handling operations that use/modify RNG #291

Open
Open
[Discussion]: Handling operations that use/modify RNG#291
Labels
corediscussion

Description

@averissimo
averissimo
opened on Aug 26, 2026

What is your question?

Per discussion with @llrs-roche, here are some topics for the contributors to discuss and research.

Problem

Random/stoichastic operations are not being well handled in within/teal.code::eval_code, in particular when code parse is being used.

In the section below you can observe that q |> get_code(names = "IRIS") is not fully reproducible as the random operations before were not included, nor was the RNG state updated before code execution

Reproducible code

This code highlights the 2 main problems:

  1. set.seed() and other random operations are not kept in partial code (q |> get_code(names = "XXX"))
  2. MTCARS/IRIS random operations are influenced by preceding code
library(teal.code)
q <- qenv() |> 
  within({
    IRIS <- datasets::iris
    MTCARS <- datasets::mtcars
    set.seed(1)
    .random <- runif(1)
    IRIS$new <- runif(nrow(IRIS))
    MTCARS$new <- runif(nrow(MTCARS))
  })

q |> get_code() |> cat()
#> iris <- datasets::iris
#> mtcars <- datasets::mtcars
#> set.seed(1)
#> .random <- runif(1)
#> iris$new <- runif(nrow(iris))
#> mtcars$new <- runif(nrow(mtcars))

q |> get_code(names = "IRIS") |> cat()
#> IRIS <- datasets::iris
#> IRIS$new <- runif(nrow(IRIS))

local({
  eval(str2expression(q |> get_code(names = "IRIS")))
  q$IRIS |> rlang::hash() |> cat("# hash qenv IRIS\n")
  IRIS |> rlang::hash() |> cat("# hash reproduced IRIS\n")
})
#> 9bba33c8ed5cd90cd13316f35f812f55 # hash qenv IRIS
#> c90a89c1814206bc19fdc32d04d1eefa # hash reproduced IRIS

q |> get_code(names = "MTCARS") |> cat()
#> MTCARS <- datasets::mtcars
#> MTCARS$new <- runif(nrow(MTCARS))

local({
  eval(str2expression(q |> get_code(names = "MTCARS")))
  q$mtcars |> rlang::hash() |> cat("# hash qenv MTCARS\n")
  mtcars |> rlang::hash() |> cat("# hash reproduced MTCARS\n")
})
#> 2c0a8a99dc147d5445c3b49d035665b2 # hash qenv MTCARS
#> 6755d143ff87b73a1196c186cec7e86a # hash reproduced MTCARS

Reference material

Possible set of solutions

  • Use teal.code::eval_code() and # @linksto XXXXX to include random operations
    • Status quo
    • Cons: Requires character base code execution and requires manual maintenance
  • Detect random seed at start and inform user
  • Track random seed and incorporate it in code execution
  • When using code parser, re-run partial code to sync modules with reproducible code (while keeping set.seed() expressions)
    • cons: slow down teal app even more and data is not consistent with initial data argument in teal.
  • Always include set.seed() expression and hope it doesn't have a big impact

Code of Conduct

  • I agree to follow this project's Code of Conduct.

Contribution Guidelines

  • I agree to follow this project's Contribution Guidelines.

Security Policy

  • I agree to follow this project's Security Policy.

Activity

  1. averissimo
    added
    discussion
    on Aug 26, 2026

  2. averissimo commented on Aug 26, 2026

    @averissimo
    averissimo
    on Aug 26, 2026
    ContributorAuthor

    My latest idea to tackle this problem is to inject a random seed state before each code LINE execution, and remove it if random seed does not change (see ref2)

    Key ideas

    • Keep indirect state of random seed (replicating exact state would be too verbose)
    • Track random seed and only keep injected code if operation has a random component

    Workflow

    Let's say if we would execute code:

    q <- teal_data() |> within({
  ADSL <- teal.data::rADSL
  ADSL <- dplyr::select(ADSL, 1:10)
  ADSL$yada <- runif(nrow(ADSL))
})

    Then teal.code would inject an indirect state of the current seed and execute in the background:

    set.seed(876990) # 876990 is generated by teal.code
ADSL <- teal.data::rADSL
set.seed(873884) # 873884 is generated by teal.code
ADSL <- dplyr::select(ADSL, 1:10)
set.seed(513737) # 513737 is generated by teal.code
ADSL$yada <- runif(nrow(ADSL))

    Discard from code set.seed() that are not needed, as internal random seed did not change

    q |> get_code() |> cat()
#> ADSL <- teal.data::rADSL
#> ADSL <- dplyr::select(ADSL, 1:10)
#> set.seed(513737)
#> ADSL$yada <- runif(nrow(ADSL))

    We would need to link every injected set.seed() with the following expression so that the injected line would be included.

    Limitations

    • Changes original code as injection of set.seed(<generated number by teal.code>) guarantees a different state
    • Adds overhead to code execution

  3. llrs-roche commented on Aug 27, 2026

    @llrs-roche
    llrs-roche
    on Aug 27, 2026
    Member

    Then teal.code would inject an indirect state of the current seed and execute in the background:

    set.seed(876990) # 876990 is generated by teal.code
    ADSL <- teal.data::rADSL
    set.seed(873884) # 873884 is generated by teal.code
    ADSL <- dplyr::select(ADSL, 1:10)
    set.seed(513737) # 513737 is generated by teal.code
    ADSL$yada <- runif(nrow(ADSL))

    I don't don't think we need to change the random seed number generation each time. The seed is fine for parallel process and doesn't change the actual numbers only the index:

    set.seed(41)
head(.Random.seed)
#> [1]      10403        624 1957627861 1533286034 1524464875 2141189936
runif(1)
#> [1] 0.2134905
head(.Random.seed)
#> [1]       10403           1   491320415  1131310133 -1634207796    93692706
runif(1)
#> [1] 0.9712283
head(.Random.seed)
#> [1]       10403           2   491320415  1131310133 -1634207796    93692706
runif(1)
#> [1] 0.578187
head(.Random.seed)
#> [1]       10403           3   491320415  1131310133 -1634207796    93692706
runif(1)
#> [1] 0.14208
head(.Random.seed)
#> [1]       10403           4   491320415  1131310133 -1634207796    93692706

    Created on 2026-08-27 with reprex v2.1.1

    The .Random.seed is filled from the main R process and it is not "available" on the qenv, based on these tests:

    library(teal.code)

set.seed(42)
qa <- within(qenv(), {
  a <- .Random.seed
})
qa |> get_code() |> cat()
#> Error:
#> ! object '.Random.seed' not found 
#>  when evaluating qenv code:
#> a <- .Random.seed
#> 
#> trace: 
#>  a <- .Random.seed
qa |> get_outputs()
#> Error:
#> ! unable to find an inherited method for function 'get_outputs' for signature 'object = "qenv.error"'

q <- within(qenv(), {
  for (i in seq_len(10)) {
    cat(runif(1), sep = "\n")
  }
})

q |> get_code() |> cat()
#> for (i in seq_len(10)) {
#>     cat(runif(1), sep = "\n")
#> }
q |> get_outputs()
#> [[1]]
#> [1] "0.2861395\n0.8304476\n0.6417455\n0.5190959\n0.7365883\n0.1346666\n0.6569923\n0.7050648\n0.4577418\n0.7191123\n"

q <- within(qenv(), {
  for (i in seq_len(10)) {
    set.seed(42)
    cat(runif(1), sep = "\n")
  }
})

q |> get_code() |> cat()
#> for (i in seq_len(10)) {
#>     set.seed(42)
#>     cat(runif(1), sep = "\n")
#> }
q |> get_outputs()
#> [[1]]
#> [1] "0.914806\n0.914806\n0.914806\n0.914806\n0.914806\n0.914806\n0.914806\n0.914806\n0.914806\n0.914806\n"

    But uses the external one:

    q |> get_code() |> cat()
#> for (i in seq_len(10)) {
#>     cat(runif(1), sep = "\n")
#> }
q |> get_outputs()
#> [[1]]
#> [1] "0.9346722\n0.2554288\n0.4622928\n0.9400145\n0.9782264\n0.1174874\n0.4749971\n0.5603327\n0.9040314\n0.1387102\n"

set.seed(42)
q <- within(qenv(), {
  for (i in seq_len(10)) {
    cat(runif(1), sep = "\n")
  }
})

q |> get_code() |> cat()
#> for (i in seq_len(10)) {
#>     cat(runif(1), sep = "\n")
#> }
q |> get_outputs()
#> [[1]]
#> [1] "0.914806\n0.9370754\n0.2861395\n0.8304476\n0.6417455\n0.5190959\n0.7365883\n0.1346666\n0.6569923\n0.7050648\n"

    But they can use the internal one:

    set.seed(42)
q4 <- within(q, {
  for (i in seq_len(10)) {
    cat(runif(1), sep = "\n")
  }
})

q5a <- within(q4, {
  for (i in seq_len(10)) {
    cat(runif(1), sep = "\n")
  }
})
q5a |> get_outputs()
#> [[1]]
#> [1] "0.914806\n0.9370754\n0.2861395\n0.8304476\n0.6417455\n0.5190959\n0.7365883\n0.1346666\n0.6569923\n0.7050648\n"
#> 
#> [[2]]
#> [1] "0.914806\n0.9370754\n0.2861395\n0.8304476\n0.6417455\n0.5190959\n0.7365883\n0.1346666\n0.6569923\n0.7050648\n"
#> 
#> [[3]]
#> [1] "0.9346722\n0.2554288\n0.4622928\n0.9400145\n0.9782264\n0.1174874\n0.4749971\n0.5603327\n0.9040314\n0.1387102\n"

q5b <- within(q4, {
  set.seed(48)
  for (i in seq_len(10)) {
    cat(runif(1), sep = "\n")
  }
})
q5b |> get_outputs()
#> [[1]]
#> [1] "0.914806\n0.9370754\n0.2861395\n0.8304476\n0.6417455\n0.5190959\n0.7365883\n0.1346666\n0.6569923\n0.7050648\n"
#> 
#> [[2]]
#> [1] "0.914806\n0.9370754\n0.2861395\n0.8304476\n0.6417455\n0.5190959\n0.7365883\n0.1346666\n0.6569923\n0.7050648\n"
#> 
#> [[3]]
#> [1] "0.5791662\n0.03382941\n0.002716767\n0.3773311\n0.243292\n0.11888\n0.9810253\n0.03831031\n0.7853011\n0.4635564\n"

    Created on 2026-08-27 with reprex v2.1.1

    The good news is that adding seeds won't slow down the report as when we save it, it runs all the code chunks again IIRC.

  4. donyunardi
    added
    core
    on Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    corediscussion

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions