Skip to content

Some clarifications needed on TransmitProfile and EventPriority #914

Closed
Closed
Some clarifications needed on TransmitProfile and EventPriority#914
Assignees
larvaceamkoscumbeduardo-camacho
Labels
iOSiOS related issuequestionFurther information is requested

Description

@nishchith-cp
Nishchith C P (nishchith-cp)
opened on Aug 17, 2021

Need these details for both iOS and Android:

  1. Can we set the transmit profile to the LogManager multiple times? Eg: Set different transmit profile based on the network strength
  2. What priority events will be skipped for NearRealTime and BestEffort transmit profiles?
 It was mentioned in the documentation that “Low priority events will not be transmitted, except for the RealTime transmit profile, unless it will not incur additional cost to the client application in terms of resource consumption, which mostly means when the device is on plug-in power and on a non-metered network”
  3. ODWEventPriorityUnspecified -> This is the default event priority being used. In what cases events will be skipped with this priority?
  4. What's the order of the event priorities for being skipped?
  5. We would want to specify custom TransmitProfile instead of the default ones provided by the SDK. As per the below documentation, we need to set the timer value seconds for the associated event priorities. We would want to control these values through ECS settings.
  1. Is the above documentation valid? Is there any OneDS documentation on this?
typedef NS_ENUM(NSInteger, ODWEventPriority)
{
    ODWEventPriorityUnspecified = -1,
    ODWEventPriorityOff = 0,
    ODWEventPriorityLow = 1,
    ODWEventPriorityNormal = 2,
    ODWEventPriorityHigh = 3,
    ODWEventPriorityImmediate = 4
};

typedef NS_ENUM(NSInteger, ODWTransmissionProfile)
{
    ODWRealTime = 0,
    ODWNearRealTime = 1,
    ODWBestEffort = 2
}

cc: aamittal1989 , karthika-msft

Activity

  1. Nishchith C P (nishchith-cp)
    added
    questionFurther information is requested
    on Aug 17, 2021
  2. Nishchith C P (nishchith-cp)
    assigned
    Sid Dahiya (sid-dahiya)
    on Aug 17, 2021
  3. Sid Dahiya (sid-dahiya)
    assigned
    Martin Harriman (larvacea)
    ,
    eduardo-camacho
    and
    Matthew Koscumb (mkoscumb)
    and unassigned
    Sid Dahiya (sid-dahiya)
    on Aug 17, 2021

  4. kindbe commented on Aug 17, 2021

    @kindbe
    David Brown (kindbe)
    on Aug 17, 2021
    Contributor

    https://github.com/microsoft/cpp_client_telemetry/blob/master/docs/custom-transmit-profile.md has some more details about decoding the transmit profiles. Others on the issue, please jump in and correct any of the below if I'm wrong:

    1. You can change the active profile at any point.
    2. The timer definitions for NearRealTime and BestEffort can be found in lib/tmp/TransmitProfiles.cpp starting on line 373. If I'm reading them correctly, in both cases they are skipping EventLatency_Normal and EventLatency_CostDeferred events on metered networks.
    3. In the case of ODWEventPriorityUnspecified, it is using the default value for the underlying native EventProperties object, which is EventLatency_Normal.
    4. "Priority" is overloaded to mean both upload latency and storage retention. Per custom-transmit-profile.md, the first digit of the profile controls EventLatency_Normal and EventLatency_CostDeferred, the second digit is not used (but should be set to the same value as the first digit), and the final digit controls the EventLatency_Realtime upload cadence.
    5. The obj-c wrapper doesn't provide a way to set a custom transmit profile; I'd recommend logging an issue (or submitting a PR). I don't know if there are formal recommendations, but I can share that we use a five minute upload cadence, and block upload for Normal/CostDeferred events on metered networks. We've tried to minimize cost to consumers both in terms of bandwidth costs and battery drain via network wakeups. In terms of testing data loss, I don't know if evt_stats captures how many events are skipped, but that might be one way. Another might be adding your own sequence number and looking for gaps.
    6. That document is out of date, and I'd still start with https://github.com/microsoft/cpp_client_telemetry/blob/master/docs/custom-transmit-profile.md.

  5. suryatn commented on Sep 23, 2021

    @suryatn
    Surya Namavaram (suryatn)
    on Sep 23, 2021

    David Brown (@kindbe) Max Golovanov (@maxgolov) Lalit Kumar Bhasin (@lalitb) Martin Harriman (@larvacea) Sid Dahiya (@sid-dahiya) Matthew Koscumb (@mkoscumb) - we have below follow up queries regarding custom transmit profile:

    1. Can we switch from custom transmit profile to built-in transmit profile and vice versa at any point in the applifecycle?
    2. In Android, we use only these 2 event latencies as per this logic: EventLatency.RealTime (for High, Immediate Priority), EventLatency.Normal (for Low, Normal Priority) i.e we do not use EventLatency.CostDeferred at all. Do we need to set the 2nd value of timer equal to the first one as per this wiki? Because as per timer definitions of built-in NearRealTime here in code, the timer values of first and second are different. We want to initially replicate exactly the built-in NearRealTime profile and want to experiment changing the timer value via ECS.
    3. What happens to the events where timer is set to -1 (e.g EventLatency.Normal in metered condition), will they be stored in-memory/cache and be uploaded once the conditions are better?
    4. Can we expect any gains in terms of perf/power/COGS by increasing the timer value in custom profile (e.g from 3 seconds for built-in NearRealTime profile in Unmetered charging to some other value say 5 or 10 seconds)?
    5. Even as per debug logs, we are seeing that the timer values are matching to that mentioned here in code. Can we get a confirmation on what should be the ideal JSON for built-in NearRealTime transmit profile i.e if we should follow as per wiki or the code?
    2021-09-23 19:48:06.013 8007-8488/com.microsoft.skype.teams.dev D/MATSDK: name=NEAR_REAL_TIME
2021-09-23 19:48:06.013 8007-8488/com.microsoft.skype.teams.dev D/MATSDK: [0] netCost= 3, powState=-1, timers=[ -1, -1, -1]
2021-09-23 19:48:06.013 8007-8488/com.microsoft.skype.teams.dev D/MATSDK: [1] netCost= 2, powState= 0, timers=[ -1, 24, 12]
2021-09-23 19:48:06.013 8007-8488/com.microsoft.skype.teams.dev D/MATSDK: [2] netCost= 2, powState= 1, timers=[ -1, 24, 12]
2021-09-23 19:48:06.013 8007-8488/com.microsoft.skype.teams.dev D/MATSDK: [3] netCost= 2, powState= 2, timers=[ -1, 18,  9]
2021-09-23 19:48:06.013 8007-8488/com.microsoft.skype.teams.dev D/MATSDK: [4] netCost= 1, powState= 0, timers=[ 24, 12,  6]
2021-09-23 19:48:06.013 8007-8488/com.microsoft.skype.teams.dev D/MATSDK: [5] netCost= 1, powState= 1, timers=[ 24, 12,  6]
2021-09-23 19:48:06.013 8007-8488/com.microsoft.skype.teams.dev D/MATSDK: [6] netCost= 1, powState= 2, timers=[ 12,  6,  3]
2021-09-23 19:48:06.013 8007-8488/com.microsoft.skype.teams.dev D/MATSDK: [7] netCost= 0, powState= 0, timers=[ 24, 12,  6]
2021-09-23 19:48:06.013 8007-8488/com.microsoft.skype.teams.dev D/MATSDK: [8] netCost= 0, powState= 1, timers=[ 24, 12,  6]
2021-09-23 19:48:06.013 8007-8488/com.microsoft.skype.teams.dev D/MATSDK: [9] netCost= 0, powState= 2, timers=[ 12,  6,  3]
2021-09-23 19:48:06.013 8007-8488/com.microsoft.skype.teams.dev D/MATSDK: [10] netCost=-1, powState=-1, timers=[ -1, -1, -1]

    Please help in clarifying these queries as we want to start experimenting with custom transmit profiles in Teams Android.

    cc aamittal1989 Nishchith C P (@nishchith-cp)

  6. lalitb commented on Sep 24, 2021

    @lalitb
    Lalit Kumar Bhasin (lalitb)
    on Sep 24, 2021
    Contributor

    Surya Namavaram (@suryatn) - I can reply to a few of your queries. Others on the issue can pitch in for other queries or correct me, else I will reply to the rest tomorrow after validating the code. Would also like to hear the result of Surya's experiments.

    1. Yes, you should be able to switch across different transmit profiles ( custom or default ) in between the app processing.
    3. Events are not lost or dropped as long as they don't cross the configured memory/DB limit. They are stored, and transmitted once the network condition, battery condition, or transmit profile is changed.
    4. Yes, theoretically you should see a gain in perf/power with an increase in the time between uploads, best would be to profile the application against different values to see the result.

  7. lalitb commented on Sep 24, 2021

    @lalitb
    Lalit Kumar Bhasin (lalitb)
    on Sep 24, 2021
    Contributor

    Surya Namavaram (@suryatn), regarding 2 and 5 queries as per the code below:

    1. If the first two timer values are equal, it would be the normal transmission irrespective of the third timer value.
    2. Else. If the first timer value is negative, it would be the real-time transmission.
    3. Else transmission latency alternates between normal and real time.

    cpp_client_telemetry/lib/tpm/TransmissionPolicyManager.cpp

    Lines 358 to 378 in 0fa60bb

    EventLatency TransmissionPolicyManager::calculateNewPriority()
    {
    updateTimersIfNecessary();
    if (m_timers[0] == m_timers[1])
    {
    return EventLatency_Normal;
    }
    if (m_timers[0] < 0)
    {
    return EventLatency_RealTime;
    }
    if (m_runningLatency == EventLatency_RealTime)
    {
    return EventLatency_Normal;
    }
    return EventLatency_RealTime;
    }

    So to achieve the near-realtime transmissions, the first two values shouldn't be equal, and the second value is ignored in code ( other than in the above comparison ). I would stick to the values in the code for near-real-time transmission.

    Tagging Martin Harriman (@larvacea) as the author of the wiki doc, if he has other/more recommedations.

  8. suryatn commented on Sep 25, 2021

    @suryatn
    Surya Namavaram (suryatn)
    on Sep 25, 2021

    Thanks Lalit Kumar Bhasin (@lalitb). We will stick to the values mentioned in the code for NearRealTime transmission for now.

    Can you also help explain the below:

    1. Else. If the first timer value is negative, it would be the real-time transmission.

    What would happen in case of Roaming rule where all the timers are [-1, -1, -1] (ref)? I am assuming even the EventLatency_RealTime events will not be sent over wire as its timer value is -1.

    1. Else transmission latency alternates between normal and real time.

    Does it mean that if the first and second timers are > 0 (ref), sometimes EventLatency_RealTime events get sent over wire and other time EventLatency_Normal events get sent and it alternates between the two? also when does it alternate?

    Tagging Martin Harriman (@larvacea) as the author of the wiki doc, if he has other/more recommedations.

    Martin Harriman (@larvacea) can you also please check once and provide your inputs? Tagging Max Golovanov (@maxgolov) incase you have some suggestions as well.

  9. suryatn commented on Sep 29, 2021

    @suryatn
    Surya Namavaram (suryatn)
    on Sep 29, 2021

    Checked with Martin Harriman (@larvacea). As per him the values mentioned in code and the format suggested in wiki should both work fine for NearRealTime transmissions. He can provide more details / context here.

  10. Ayushi Kejriwal (aykejriw)
    added
    iOSiOS related issue
    on Jan 4, 2022
  11. microsoft
    locked and limited conversation to collaborators on Jun 30, 2022
  12. Lalit Kumar Bhasin (lalitb)
    converted this issue into a discussion #1031 on Jun 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

iOSiOS related issuequestionFurther information is requested

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions