PR: Reduce calls to get environment variables in SpyderKernelSpec and raise timeout threshold to do that

[mrclary]

Description of Changes

I propose raising the timeout to 5s for all scenarios.
This PR raises the timeout for the subprocess for getting user environment variables from 0.5s to 5s, and reduces superfulous calls to get_user_environment_variables by moving it to the instantiation of SpyderKernelSpec from SpyderKernelSpec.env.

I've run into a scenario where get_user_environment_variables can take up to ~1s or more. This triggers the timeout and renders the function useless, leaving users with incorrect environment variables sent to the console and language server.

Currently, get_user_environment_variables is called in SpyderKernelSpec.env, which is not appropriate since this is decorated with @property. SpyderKernelSpec.env can be referenced many times, resulting in superfluous exectution of get_user_environment_variables (8 times during Spyder startup!).

[mrclary]

@ccordoba12, I think you mentioned using spyder.api.asyncdispatcher for this PR; was that correct? If so, I may need some guidance on that.

[mrclary]

#21769 introduced running the user_env.sh script in pytests. Without it, get_user_environment_variables cannot be fully tested on CI. While this works on CI, locally run pytests (at least for me) that use the ipyconsole fixture get a SIGSTP somehow. Probably because local tests are run in an interactive shell, while CI is not (see #22415). Exchanging running_under_pytest for running_in_ci should fix this.

Now, specific to this PR, failing tests:

• test_old_kernel_version: kernel_handler.check_spyder_kernel_info(('1.0.0', '')) appears ineffective; a new client is successfully created
• test_dedicated_consoles: appears to be a race condition. Sometimes qtbot.waitUntil(lambda: nsb.editor.source_model.rowCount() == 4) times out even though there are 4 entries; sometimes a %clear command is issued with the F5 key so that the banner is not captured and assert ('runfile' in text) and ('Python' in text and 'IPython' in text) fails. 🤷🏼 . Perhaps moving get_user_environment_variables out of the property SpyderKernelSpec.env changed its performance (for the better) but then messed with the timing in this test.

I don't believe there is any issue with moving get_user_environment_variables. I think the issue is elsewhere. Perhaps a race condition with some Qt signals in the run or editor plugins, and the kernel handler.

[mrclary]

The reason the tests were failing was due to checking the kernel cache, which compares all attributes of the SpyderKernelSpec. The new private _env_vars attribute was causing the comparison to fail and the solution was to ensure that PYTEST_CURRENT_TEST key was removed from _env_vars.

test_dedicated_console still has issues on my local macOS machine, but not on my Linux VM, so it may be macOS intel related. Nevertheless, it seems to be working on CI now, so I'll leave well enough alone.

[ccordoba12]

A couple of small suggestions for you @mrclary, the rest looks good to me.

[ccordoba12]

Looks good to me now, thanks @mrclary!

[ccordoba12]

@meeseeksdev please backport to 6.x

[lumberbot-app]

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

git checkout 6.x
git pull

2. Cherry pick the first parent branch of the this PR on top of the older branch:

git cherry-pick -x -m1 9314704ed5b6bca4b75ba7f41b849349fb8c97c0

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

git commit -am 'Backport PR #23278: PR: Reduce calls to get environment variables in `SpyderKernelSpec` and raise timeout threshold to do that '

4. Push to a named branch:

git push YOURFORK 6.x:auto-backport-of-pr-23278-on-6.x

5. Create a PR against branch 6.x, I would have named this PR:

"Backport PR #23278 on branch 6.x (PR: Reduce calls to get environment variables in SpyderKernelSpec and raise timeout threshold to do that )"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.