Skip to content

Issue 3211 fix flaky tests part 2 - #3347

Open
DenBond7 wants to merge 20 commits into
masterfrom
issue_3211_fix_flaky_tests_part_2
Open

DenBond7 wants to merge 20 commits into
masterfrom
issue_3211_fix_flaky_tests_part_2

Conversation

@DenBond7

@DenBond7 DenBond7 commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

This PR fixes more flaky instrumentation tests that fail on slow CI emulators (part 2, follows #3340) and re-enables 3 of them.

Most failures are races with recipient lookups and lost key events on the compose screen:

  • Lost characters in typeText(). On a slow emulator the IME restarts its input session and the app drops text committed to the old session (Session id mismatch ... commitText in logcat), e.g. "message" -> "essage" or a recipient ...@flowcrypt.te.
    • ci-wait-for-emulator.sh now disables the IME (Gboard and the Google TTS voice IME) on test emulators. DISABLE_IME=0 keeps the keyboard.
    • A new insertTextAtCursor() view action inserts text without key events. BaseComposeScreenTest.fillInAllFields() uses it for the message body.
    • Tests use replaceText() instead of typeText() for recipients, subject and message.
  • Sending/checking before the recipient lookup is done. While the lookup is in progress the chip has the "updating" color and sending is blocked with a "please wait" toast. Tests now call waitUntilRecipientChipIsSettled() instead of a single chip check or Thread.sleep().

Fixed tests:

  • ComposeScreenFlowTest#testEmptyEmailSubject: fixed and re-enabled
  • ComposeScreenFlowTest#testSelectImportPublicKeyFromPopUp: fixed and re-enabled (also replaced a 5 × Thread.sleep(1000) clipboard loop with waitUntil)
  • CreateMessageFragmentDisallowAttesterSearchInIsolationTest#testDisallowLookupOnAttester: fixed and re-enabled
  • ComposeScreenFlowTest#testSelectedRemoveRecipientFromPopUp, #testSelectedStandardEncryptionTypeFromPopUp, #testSelectedCopyFromOtherContactFromPopUp
  • CreateMessageFragmentDisallowAttesterSearchForDomainInIsolationTest#testCanLookupThisRecipientOnAttester
  • PrivateKeyDetailsFragmentPassInRamFlexibleTimeFlowTest#testInMemoryPassPhraseSessionLengthParameter: waits until the pass phrase expires instead of a fixed Thread.sleep(DELAY_TIMEOUT). The service can start late on a slow emulator.

Test infrastructure:

  • RetryRule prints the stack trace of every failed run, so failures hidden by a successful retry are visible in logcat.
  • ScreenshotTestRule takes screenshots via UiAutomation. androidx.test takeScreenshot() force-redraws app windows and could crash the app process (NPE in ViewCapture.forceRedraw) when a window was being destroyed, which hid the original failure and stopped RetryRule. A screenshot error no longer replaces the original failure.
  • FileAndDirectoryUtils: a file removed by another component (e.g. Glide's disk cache) during directory cleanup is not reported as an error.
  • docs/testing-like-ci.md: how to repeat a test from a class with a @ClassRule mock web server (RepeatableAndroidJUnit4ClassRunner starts the rule twice and fails with EADDRINUSE).

Verified on a CI-like slow emulator (docs/testing-like-ci.md, RetryRule(1)): e.g. testEmptyEmailSubject failed 4 of 5 runs before the fix and passed 15 of 15 after it. The whole ComposeScreenFlowTest class passed without retries.

issue #3211


Tests (delete all except exactly one):

  • Tests added or updated

To be filled by reviewers

I have reviewed that this PR... (tick whichever items you personally focused on during this review):

  • addresses the issue it closes (if any)
  • code is readable and understandable
  • is accompanied with tests, or tests are not needed
  • is free of vulnerabilities

DenBond7 and others added 20 commits November 13, 2026 21:58
Glide's disk cache renames its journal while it rebuilds it. When the
file disappears between exists() and delete(), delete() returns false
and cleanDir() threw IOException. This broke ClearAppSettingsRule
(e.g. WkdClientTest on CI). Treat it as an error only when the file
still exists.
androidx.test takeScreenshot() redraws the app windows on the main
thread. When a window is being destroyed after a failure, it crashes the
app process with NPE in ViewCapture.forceRedraw. The crash hides the
original failure and stops RetryRule. UiAutomation takes the screenshot
on the system side, and any screenshot error is now only logged.
RetryRule printed only "run N failed". Print the stack trace too, so a
failure hidden by a successful retry can be found in logcat.
On a slow emulator the IME session restarts after the focus change and
typed characters are lost ("message" became "essage" in
EncryptedReplyComposeGmailApiFlow). fillInAllFields() now inserts the
message at the cursor directly and keeps existing text, e.g. a
signature.
PassPhrasesInRAMService clears expired pass phrases once per
DELAY_TIMEOUT, counted from the service start, and the screen is updated
asynchronously. Wait for the result instead of one check after a fixed
sleep.
- Clicking Send during the recipient lookup shows a "please wait" toast
  instead of the dialog. The popup tests now wait until the recipient
  chip is settled.
- The chip color check in
  CreateMessageFragmentDisallowAttesterSearchForDomainInIsolationTest
  now waits for the lookup result.
- typeText() lost characters when the IME session restarted (the
  recipient became "...@flowcrypt.te"), so these tests use
  replaceText().
On slow emulators Gboard restarts its input session after focus
changes, and the app drops text committed to the old session
("Session id mismatch ... commitText" in logcat). typeText() then loses
characters, e.g. "message" became "essage" and "...@flowcrypt.test"
became "...@flowcrypt.te".

ci-wait-for-emulator.sh now disables Gboard and the Google TTS voice
IME (the system falls back to it without Gboard), so Espresso key
events go to the view directly. On a CI-like emulator 44 typeText-heavy
tests ran in ~622s without an IME vs 680-813s with Gboard, with no new
failures. Set DISABLE_IME=0 to keep the keyboard.
# Conflicts:
#	FlowCrypt/src/androidTest/java/com/flowcrypt/email/base/BaseTest.kt
#	FlowCrypt/src/androidTest/java/com/flowcrypt/email/ui/gmailapi/passwordprotected/BaseComposeScreenPasswordProtectedDisallowedTermsTest.kt
#	FlowCrypt/src/androidTest/java/com/flowcrypt/email/ui/gmailapi/passwordprotected/ComposeScreenPasswordProtectedDisallowedTermsReFetchConfigurationFlowTest.kt
#	script/ci-wait-for-emulator.sh
@DenBond7
DenBond7 marked this pull request as ready for review November 19, 2026 13:09
@DenBond7
DenBond7 requested a review from sosnovsky as a code owner November 19, 2026 13:09
@DenBond7 DenBond7 mentioned this pull request Nov 19, 2026
15 of 52 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant