-
Notifications
You must be signed in to change notification settings - Fork 297
Add MAUI Android inner loop deploy measurement scenario #5165
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
davidnguyen-tech
wants to merge
27
commits into
dotnet:main
Choose a base branch
from
davidnguyen-tech:feature/measure-maui-android-deploy
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
8123eed
Add AndroidInnerLoopParser for build/deploy binlog analysis
davidnguyen-tech f6c5869
Add ANDROIDINNERLOOP test type to Python test framework
davidnguyen-tech 7740568
Add MAUI Android inner loop scenario scripts
davidnguyen-tech 3f5e643
Wire MAUI Android inner loop into CI pipeline
davidnguyen-tech 746709a
Make screen timeout configurable and validate cold startup LaunchState
davidnguyen-tech 986c7bf
Address review notes
davidnguyen-tech 26ea3ea
Address review notes 2
davidnguyen-tech 59d3d40
Add SDK version capture + diagnostics to mauiandroidinnerloop
davidnguyen-tech efd2217
Address copilot review notes
davidnguyen-tech d73f5fb
Log rollback_maui.json contents on Helix machine
davidnguyen-tech d5da6c1
Switch MAUI Android Inner Loop to 'dotnet run -p:WaitForExit=false'
davidnguyen-tech c47d731
Migrate MAUI Android Inner Loop startup measurement to logcat polling
davidnguyen-tech 7c1e3bf
Remove stale Android cold startup helper
davidnguyen-tech e2abd8d
Address PR review findings
davidnguyen-tech 27ef49a
Reject non-Debug BuildConfig for MAUI Android Inner Loop
davidnguyen-tech 57d68ff
Address PR review: align with Android verifier-disable + clarify semi…
davidnguyen-tech 0f1a76c
Revert verifier-disable change; keep semicolon-list help text
davidnguyen-tech d019e81
Revert ValidateInnerLoopBuildConfig guard and -c $(BuildConfig) switch
davidnguyen-tech 802b12f
Document why -c Debug is hardcoded in inner loop scenario
davidnguyen-tech b03125d
MAUI Android Inner Loop: pass AndroidManifestType=GoogleV2 for API 37
davidnguyen-tech dffb8fc
[mauiandroidinnerloop] Repair silently-skipped Android SDK deps
davidnguyen-tech 362bf00
Consolidate inner-loop proj into maui_scenarios_android.proj
kotlarmilos 530503f
Pass RunKind to Send job to Helix step
kotlarmilos 053f80c
Guard Helix traces cleanup on non-null upload dir
kotlarmilos 7e3e8e3
Address review feedback: source restore, adb timeout, PEP 604 compat
kotlarmilos 7ec125c
androidhelper: use grep -F so dots in package name aren't regex wildc…
kotlarmilos b257e9c
Address review feedback on PR #5165
kotlarmilos File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| <Project Sdk="Microsoft.DotNet.Helix.Sdk" DefaultTargets="Test"> | ||
|
|
||
| <Import Project="Scenarios.Common.props" /> | ||
|
|
||
| <PropertyGroup> | ||
| <IncludeXHarnessCli>true</IncludeXHarnessCli> | ||
|
|
||
| <_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'mono'">/p:UseMonoRuntime=true</_MSBuildArgs> | ||
| <_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'coreclr'">/p:UseMonoRuntime=false;/p:PublishReadyToRun=false;/p:PublishReadyToRunComposite=false</_MSBuildArgs> | ||
|
|
||
| <AndroidRid Condition="'$(AndroidRid)' == '' and '$(TargetsWindows)' == 'true'">android-arm64</AndroidRid> | ||
| <AndroidRid Condition="'$(AndroidRid)' == '' and '$(TargetsWindows)' != 'true'">android-x64</AndroidRid> | ||
| <_MSBuildArgs>$(_MSBuildArgs) /p:RuntimeIdentifier=$(AndroidRid)</_MSBuildArgs> | ||
|
|
||
| <!-- https://github.com/dotnet/maui/issues/34706 --> | ||
| <_MSBuildArgs>$(_MSBuildArgs) /p:SupportedOSPlatformVersion=23</_MSBuildArgs> | ||
| <_MSBuildArgs Condition="!$(_MSBuildArgs.Contains('/p:TargetFrameworks='))">$(_MSBuildArgs) /p:TargetFrameworks=$(PERFLAB_Framework)-android</_MSBuildArgs> | ||
|
|
||
| <RunConfigsString>$(RuntimeFlavor)_$(CodegenType)</RunConfigsString> | ||
| <InnerLoopIterations Condition="'$(InnerLoopIterations)' == ''">10</InnerLoopIterations> | ||
| <ScreenTimeoutMs Condition="'$(ScreenTimeoutMs)' == ''">1800000</ScreenTimeoutMs> | ||
| </PropertyGroup> | ||
|
|
||
| <Target Name="RemoveDotnetFromCorrelationStaging" BeforeTargets="BeforeTest"> | ||
| <Message Text="Removing Dotnet Packs from Correlation Staging" Importance="high" /> | ||
| <RemoveDir Directories="$(CorrelationPayloadDirectory)dotnet\packs" Condition="'$(TargetsWindows)' == 'true'" /> | ||
| <RemoveDir Directories="$(CorrelationPayloadDirectory)dotnet/packs" Condition="'$(TargetsWindows)' != 'true'" /> | ||
| </Target> | ||
|
|
||
| <ItemDefinitionGroup> | ||
| <HelixWorkItem> | ||
| <Timeout>01:00</Timeout> | ||
| </HelixWorkItem> | ||
| </ItemDefinitionGroup> | ||
|
|
||
| <ItemGroup> | ||
| <MAUIAndroidInnerLoopScenario Include="MAUI Android Inner Loop"> | ||
| <ScenarioDirectoryName>mauiandroidinnerloop</ScenarioDirectoryName> | ||
| <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory> | ||
| </MAUIAndroidInnerLoopScenario> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <PreparePayloadWorkItem Include="@(MAUIAndroidInnerLoopScenario)"> | ||
| <Command>$(Python) pre.py default -f $(PERFLAB_Framework)</Command> | ||
| <WorkingDirectory>%(PreparePayloadWorkItem.PayloadDirectory)</WorkingDirectory> | ||
| </PreparePayloadWorkItem> | ||
| </ItemGroup> | ||
|
|
||
| <PropertyGroup> | ||
| <_WindowsEnvVars>set DOTNET_ROOT=%HELIX_CORRELATION_PAYLOAD%\dotnet;set DOTNET_CLI_TELEMETRY_OPTOUT=1;set DOTNET_MULTILEVEL_LOOKUP=0;set NUGET_PACKAGES=%HELIX_WORKITEM_ROOT%\.packages;set ANDROID_HOME=%HELIX_WORKITEM_ROOT%\android-sdk;set ANDROID_SDK_ROOT=%HELIX_WORKITEM_ROOT%\android-sdk;set JAVA_HOME=%HELIX_WORKITEM_ROOT%\jdk;set PATH=%HELIX_CORRELATION_PAYLOAD%\dotnet%3B%HELIX_WORKITEM_ROOT%\android-sdk\platform-tools%3B%HELIX_WORKITEM_ROOT%\jdk\bin%3B%PATH%</_WindowsEnvVars> | ||
| <_LinuxEnvVars>export DOTNET_ROOT=$HELIX_CORRELATION_PAYLOAD/dotnet;export DOTNET_CLI_TELEMETRY_OPTOUT=1;export DOTNET_MULTILEVEL_LOOKUP=0;export NUGET_PACKAGES=$HELIX_WORKITEM_ROOT/.packages;export ANDROID_HOME=$HELIX_WORKITEM_ROOT/android-sdk;export ANDROID_SDK_ROOT=$HELIX_WORKITEM_ROOT/android-sdk;export JAVA_HOME=$HELIX_WORKITEM_ROOT/jdk;export ANDROID_SERIAL=emulator-5554;export PATH=$HELIX_CORRELATION_PAYLOAD/dotnet:$HELIX_WORKITEM_ROOT/android-sdk/platform-tools:$HELIX_WORKITEM_ROOT/jdk/bin:$PATH</_LinuxEnvVars> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup Condition="'$(TargetsWindows)' == 'true'"> | ||
| <HelixWorkItem Include="@(MAUIAndroidInnerLoopScenario -> 'Inner Loop Device - %(Identity)')"> | ||
| <PreCommands>$(_WindowsEnvVars);$(Python) setup_helix.py $(PERFLAB_Framework)-android "$(_MSBuildArgs)"</PreCommands> | ||
| <Command>$(Python) test.py androidinnerloop --csproj-path app/MauiAndroidInnerLoop.csproj --edit-src "src/MainPage.xaml.cs;src/MainPage.xaml" --edit-dest "app/Pages/MainPage.xaml.cs;app/Pages/MainPage.xaml" --package-name com.companyname.mauiandroidinnerloop -f $(PERFLAB_Framework)-android -c Debug --msbuild-args "$(_MSBuildArgs)" --scenario-name "%(Identity)" --inner-loop-iterations $(InnerLoopIterations) --screen-timeout-ms $(ScreenTimeoutMs) $(ScenarioArgs)</Command> | ||
| <PostCommands>$(Python) post.py</PostCommands> | ||
| <DownloadFilesFromResults>output.log</DownloadFilesFromResults> | ||
| </HelixWorkItem> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup Condition="'$(TargetsWindows)' != 'true'"> | ||
| <HelixWorkItem Include="@(MAUIAndroidInnerLoopScenario -> 'Inner Loop Emulator - %(Identity)')"> | ||
| <PreCommands>$(_LinuxEnvVars);$(Python) setup_helix.py $(PERFLAB_Framework)-android "$(_MSBuildArgs)"</PreCommands> | ||
| <Command>$(Python) test.py androidinnerloop --csproj-path app/MauiAndroidInnerLoop.csproj --edit-src "src/MainPage.xaml.cs;src/MainPage.xaml" --edit-dest "app/Pages/MainPage.xaml.cs;app/Pages/MainPage.xaml" --package-name com.companyname.mauiandroidinnerloop -f $(PERFLAB_Framework)-android -c Debug --msbuild-args "$(_MSBuildArgs)" --scenario-name "%(Identity)" --inner-loop-iterations $(InnerLoopIterations) --screen-timeout-ms $(ScreenTimeoutMs) $(ScenarioArgs)</Command> | ||
| <PostCommands>$(Python) post.py</PostCommands> | ||
| <DownloadFilesFromResults>output.log</DownloadFilesFromResults> | ||
| </HelixWorkItem> | ||
| </ItemGroup> | ||
|
|
||
| <Import Project="PreparePayloadWorkItems.targets" /> | ||
|
|
||
| </Project> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| ''' | ||
| post cleanup script | ||
| ''' | ||
|
|
||
| import subprocess | ||
| import sys | ||
| import traceback | ||
| from performance.logger import setup_loggers, getLogger | ||
| from shared.postcommands import clean_directories | ||
| from shared.util import xharness_adb | ||
| from test import EXENAME | ||
|
|
||
| setup_loggers(True) | ||
| logger = getLogger(__name__) | ||
|
|
||
| try: | ||
| # Uninstall the app from the connected device so re-runs start from a clean state | ||
| package_name = f'com.companyname.{EXENAME.lower()}' | ||
| logger.info(f"Uninstalling {package_name} from device") | ||
| subprocess.run(xharness_adb() + ['uninstall', package_name], check=False) | ||
|
|
||
| logger.info("Shutting down dotnet build servers") | ||
| subprocess.run(['dotnet', 'build-server', 'shutdown'], check=False) | ||
|
|
||
| clean_directories() | ||
| except Exception as e: | ||
| logger.error(f"Post cleanup failed: {e}\n{traceback.format_exc()}") | ||
| sys.exit(1) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,131 @@ | ||
| ''' | ||
| pre-command: Set up a MAUI Android app for deploy measurement. | ||
| Creates the template (without restore) and prepares the modified file for incremental deploy. | ||
| NuGet packages are restored on the Helix machine, not shipped in the payload. | ||
| ''' | ||
| import os | ||
| import shutil | ||
| import sys | ||
| from performance.logger import setup_loggers, getLogger | ||
| from shared import const | ||
| from shared.mauisharedpython import install_latest_maui, MauiNuGetConfigContext | ||
| from shared.precommands import PreCommands | ||
| from test import EXENAME | ||
|
|
||
| setup_loggers(True) | ||
| logger = getLogger(__name__) | ||
| logger.info("Starting pre-command for MAUI Android deploy measurement") | ||
|
|
||
| precommands = PreCommands() | ||
|
|
||
| with MauiNuGetConfigContext(precommands.framework): | ||
| install_latest_maui( | ||
| precommands, | ||
| workloads=["microsoft.net.sdk.android"], | ||
| workload_name='maui-android', | ||
| ) | ||
|
|
||
| # Log the generated rollback file for diagnostics. install_latest_maui | ||
| # writes this; if it's missing the install failed and we want to crash. | ||
| with open("rollback_maui.json", "r") as f: | ||
| logger.info(f"Generated rollback_maui.json contents:\n{f.read()}") | ||
|
|
||
| precommands.print_dotnet_info() | ||
|
|
||
| # Create template without restoring packages — packages will be restored | ||
| # on the Helix machine to avoid shipping ~1-2GB in the workitem payload. | ||
| precommands.new(template='maui', | ||
| output_dir=const.APPDIR, | ||
| bin_dir=const.BINDIR, | ||
| exename=EXENAME, | ||
| working_directory=sys.path[0], | ||
| no_restore=True, | ||
| extra_args=['-sc']) | ||
|
|
||
| # Copy the merged NuGet.config into the app directory. This file contains | ||
| # MAUI NuGet feed URLs added by MauiNuGetConfigContext. The Helix machine | ||
| # needs these feeds during restore, and we must copy before the context | ||
| # manager restores the original NuGet.config. | ||
| repo_root = os.path.normpath(os.path.join(sys.path[0], '..', '..', '..')) | ||
| repo_nuget_config = os.path.join(repo_root, 'NuGet.config') | ||
| app_nuget_config = os.path.join(const.APPDIR, 'NuGet.config') | ||
| shutil.copy2(repo_nuget_config, app_nuget_config) | ||
| logger.info(f"Copied merged NuGet.config from {repo_nuget_config} to {app_nuget_config}") | ||
|
|
||
| # Inject properties into the csproj so they apply to every command that | ||
| # targets this project (restore, build, install). | ||
| csproj_path = os.path.join(const.APPDIR, f'{EXENAME}.csproj') | ||
| with open(csproj_path, 'r') as f: | ||
| csproj_content = f.read() | ||
|
|
||
| logger.info(f"Original .csproj content:\n{csproj_content}") | ||
|
|
||
| injected_props = { | ||
| # Preview SDKs may lack prune-package-data files, causing NETSDK1226. | ||
| 'AllowMissingPrunePackageData': 'true', | ||
| # The perf repo globally disables the Roslyn compiler server to avoid | ||
| # BenchmarkDotNet file-locking issues. Re-enable it here to match real | ||
| # MAUI developer inner loop experience. | ||
| 'UseSharedCompilation': 'true', | ||
|
davidnguyen-tech marked this conversation as resolved.
|
||
| } | ||
| csproj_modified = csproj_content | ||
| if '</PropertyGroup>' not in csproj_modified: | ||
| raise Exception( | ||
| f"Cannot inject properties into {csproj_path}: " | ||
| f"no <PropertyGroup> found in the generated template." | ||
| ) | ||
| for prop_name, prop_value in injected_props.items(): | ||
| if prop_name not in csproj_modified: | ||
| csproj_modified = csproj_modified.replace( | ||
| '</PropertyGroup>', | ||
| f' <{prop_name}>{prop_value}</{prop_name}>\n </PropertyGroup>', | ||
| 1 # only the first PropertyGroup | ||
| ) | ||
|
|
||
| with open(csproj_path, 'w') as f: | ||
| f.write(csproj_modified) | ||
|
|
||
| logger.info(f"Updated {csproj_path} with injected properties") | ||
| logger.info(f"Modified .csproj content:\n{csproj_modified}") | ||
|
|
||
| # Create modified source files in src/ for the incremental deploy simulation. | ||
| # The runner toggles between original and modified versions each iteration, | ||
| # exercising both the C# compiler (Csc) and XAML compiler (XamlC) paths. | ||
| src_dir = os.path.join(sys.path[0], const.SRCDIR) | ||
| os.makedirs(src_dir, exist_ok=True) | ||
|
|
||
| # --- Modified MainPage.xaml.cs: add a debug line in the constructor --- | ||
| cs_original = os.path.join(const.APPDIR, 'Pages', 'MainPage.xaml.cs') | ||
| cs_modified = os.path.join(src_dir, 'MainPage.xaml.cs') | ||
|
|
||
| with open(cs_original, 'r') as f: | ||
| cs_content = f.read() | ||
|
|
||
| cs_modified_content = cs_content.replace( | ||
| 'InitializeComponent();', | ||
| 'InitializeComponent();\n\t\tSystem.Diagnostics.Debug.WriteLine("incremental-touch");' | ||
| ) | ||
| if cs_modified_content == cs_content: | ||
| raise Exception("Could not find 'InitializeComponent();' in %s — template may have changed" % cs_original) | ||
|
|
||
| with open(cs_modified, 'w') as f: | ||
| f.write(cs_modified_content) | ||
| logger.info(f"Modified MainPage.xaml.cs written to {cs_modified}") | ||
|
|
||
| # --- Modified MainPage.xaml: change a label's text --- | ||
| xaml_original = os.path.join(const.APPDIR, 'Pages', 'MainPage.xaml') | ||
| xaml_modified = os.path.join(src_dir, 'MainPage.xaml') | ||
|
|
||
| with open(xaml_original, 'r') as f: | ||
| xaml_content = f.read() | ||
|
|
||
| xaml_modified_content = xaml_content.replace( | ||
| 'Text="Task Categories"', | ||
| 'Text="Task Categories (updated)"' | ||
| ) | ||
| if xaml_modified_content == xaml_content: | ||
| raise Exception("Could not find 'Text=\"Task Categories\"' in %s — template may have changed" % xaml_original) | ||
|
|
||
| with open(xaml_modified, 'w') as f: | ||
| f.write(xaml_modified_content) | ||
| logger.info(f"Modified MainPage.xaml written to {xaml_modified}") | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.