镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

[wip][fix][fn] Add function name and id to producer name to avoid conflict - #21912

Closed
jiangpengcheng wants to merge 1 commit into
apache:masterfrom
jiangpengcheng:avoid_conflict_producer_name
Closed

jiangpengcheng wants to merge 1 commit into
apache:masterfrom
jiangpengcheng:avoid_conflict_producer_name

Conversation

@jiangpengcheng

Copy link
Copy Markdown
Contributor

Fixes #21911

Main Issue: #xyz

PIP: #xyz

Motivation

there will be producer name conflicts error when function enable effectively_once and subscribed to a partitioned input topic

Modifications

Add functionId and instanceId to the suffix of the producer name to avoid conflicts

Verifying this change

  • Make sure that the change passes the CI checks.
  • [ ]
  • This change is a trivial rework / code cleanup without any test coverage.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository: jiangpengcheng#21

@graysonzeng

Copy link
Copy Markdown
Contributor

Thanks for your fix. I have a question. After adding the instance name to producerName, can the effectively_once be guaranteed? For example, there was no ack for the message before the failure, but the message was resent after switching to another instance. I think deduplication is invalid at this time because of different producerNames. @jiangpengcheng

@jiangpengcheng

Copy link
Copy Markdown
Contributor Author

@graysonzeng hm, you are right, let's consider an another way to fix the issue

@Technoboy- Technoboy- closed this Jan 21, 2024
@Technoboy- Technoboy- reopened this Jan 21, 2024
@Technoboy- Technoboy- modified the milestones: 3.3.0, 3.2.0 Jan 21, 2024
@codecov-commenter

codecov-commenter commented Jan 21, 2024 •

Copy link
Copy Markdown

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (24c927e) 73.60% compared to head (247c4af) 72.39%.
Report is 12 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #21912      +/-   ##
============================================
- Coverage     73.60%   72.39%   -1.21%     
+ Complexity    32359    32030     -329     
============================================
  Files          1859     1861       +2     
  Lines        138373   143467    +5094     
  Branches      15160    16278    +1118     
============================================
+ Hits         101853   103870    +2017     
- Misses        28640    31483    +2843     
- Partials       7880     8114     +234     
Flag Coverage Δ
inttests 24.28% <0.00%> (+0.11%) ⬆️
systests 23.67% <0.00%> (-0.06%) ⬇️
unittests 71.53% <75.00%> (-1.35%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
...a/org/apache/pulsar/functions/sink/PulsarSink.java 73.75% <75.00%> (-0.10%) ⬇️

... and 113 files with indirect coverage changes

@jiangpengcheng jiangpengcheng changed the title [fix][fn] Add function name and id to producer name to avoid conflict [wip][fix][fn] Add function name and id to producer name to avoid conflict Jan 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Functions failed to recover after a crash if the input topic is partitioned and function is using EFFECTIVELY_ONCE

6 participants