adding preliminary AMDGPU support#99
Conversation
|
Thanks for doing this. I think this is the right idea and from a quick look the implementation seems good. Let me know when it is ready for review. |
|
In principle, the PR runs now, but I am getting some scalar indexing in the tests somewhere that I need to fix. I've been noticing scalar indexing in other Molly runs for a while now, but haven't been able to pin down where they are happening or if the issue is amd-specific, so I'll keep playing around here and try to find the problem. |
It seems to be here with the |
|
I have noticed scalar indexing occasionally when using Molly too, though it is disallowed in the current tests so I think that case is AMD-specific. I am not against allowing scalar indexing in specified places if performance isn't too bad. The |
|
Since the code runs, but tests are not quite passing, I think it's best to keep this PR open and refactor it into a KernelAbstractions PR on top of the |
|
Sounds good. |
|
Closing as it is outdated and #147 supersedes it. |
I am still running the tests, so this PR is still a draft and might change in the next few days, but it adds preliminary support for AMDGPU devices.
There were a few small things to note:
CuArrayto use an Array TypeATvalue instead. There were a few places (such as inzygote.jlwith internal functions that were only used for the GPU. In these cases,ATbecame a function argument. In most cases,ATcould be created without any change to the function calls, themselves.interactions/implicit_solver.jl), I added a new conditional withROCArrayinstead ofCuArraySystem(...)call was modified to try and figure out what device you wanted to use if you setgputo true, but will default toCUDAif both GPUs are present and things are left unspecifiedrun_cuda_testsandrun_rocm_tests. It will then populate the arraygpu_array_typeswithCuArray,ROCArray, or both. I tried to modify all the tests to use the array types from this array.Again, still messing around with things now, but thought I would put the draft PR up in the case that there are design decisions to talk about.