Good afternoon HCP experts,
I have been trying to run tests for preprocessing diffusion data on the HCP for Early Psychosis using the DiffusionPreprocessingBatch.sh script. I was able to successfully execute the script up to running the eddy command. Inspection of the error file revealed that optional eddy arguments that were not set in the batch processing script were passed as empty values to those arguments upon parsing to run_eddy.sh. To fix this, and to refine the action of the script, I added a series of --extra-eddy-arg arguments to the script so that empty values would no longer prevent eddy from running, since the assignment of default values seems not to be working, albeit only for certain parameters.
Note I also added some arguments available in the DiffPreprocPipeline.sh script to help refine the preprocessing steps further. The only other argument I added was --ensure-even-slices.
Since doing this, the script will not run. Upon inspection as to why, it seems something in my specification of these new arguments has broken the pipeline's argument parser, resulting in the failure to override internal variables (e.g., creation of the rawdata/ and eddy/ directories and population with their contents). As a result, this seems to have also broken my ability to override the default naming conventions of the --posData and --negData fields.
I have attached my script to this post to give a more complete picture, but for ease I have detailed the addition of new arguments here:
"${queuing_command[@]}" "${HCPPIPEDIR}"/DiffusionPreprocessing/DiffPreprocPipeline.sh \ --posData="${PosData}" \ --negData="${NegData}" \ --path="${StudyFolder}" \ --subject="${SubjectID}" \ --echospacing-seconds="${EchoSpacingSec}" \ --PEdir="${PEdir}" \ --gdcoeffs="${Gdcoeffs}" \ --printcom="$PRINTCOM" \ --ensure-even-slices=TRUE \ --extra-eddy-arg=--gpu=TRUE \ --extra-eddy-arg=--nvoxhp=2000 \ --extra-eddy-arg=--ff=10 \ --extra-eddy-arg=--resamp=lsr \ --extra-eddy-arg=--ol_nstd=4 \ --extra-eddy-arg=--peas=TRUE \ --extra-eddy-arg=--cuda-version=10.2 \ --extra-eddy-arg=--detailed_outlier-stats=TRUE \ --extra-eddy-arg=--rms=TRUE \ --extra-eddy-arg=--fwhm=0
Note a separate but related concern I have is whether the script is pulling the preprocessed T1w images from the correct location. I suspect it is not, since the data in the study folder variable (e.g., StudyFolder="/ZPOOL/data/projects/hcp-ep/imagingcollection01") does not include a folder for preprocessed T1w. I am wondering if this is a separate variable I need to specify, or if I need to replace the unprocessed T1w directory located in StudyFolder with that containing the preprocessed files. If the latter is the case, is it acceptable to move the whole preprocessed directory, or do I need to move a single file?
Kind regards,
Linda
DiffusionPreprocessingBatch.txt
Good afternoon HCP experts,
I have been trying to run tests for preprocessing diffusion data on the HCP for Early Psychosis using the
DiffusionPreprocessingBatch.shscript. I was able to successfully execute the script up to running the eddy command. Inspection of the error file revealed that optional eddy arguments that were not set in the batch processing script were passed as empty values to those arguments upon parsing torun_eddy.sh. To fix this, and to refine the action of the script, I added a series of--extra-eddy-argarguments to the script so that empty values would no longer prevent eddy from running, since the assignment of default values seems not to be working, albeit only for certain parameters.Note I also added some arguments available in the
DiffPreprocPipeline.shscript to help refine the preprocessing steps further. The only other argument I added was--ensure-even-slices.Since doing this, the script will not run. Upon inspection as to why, it seems something in my specification of these new arguments has broken the pipeline's argument parser, resulting in the failure to override internal variables (e.g., creation of the rawdata/ and eddy/ directories and population with their contents). As a result, this seems to have also broken my ability to override the default naming conventions of the
--posDataand--negDatafields.I have attached my script to this post to give a more complete picture, but for ease I have detailed the addition of new arguments here:
"${queuing_command[@]}" "${HCPPIPEDIR}"/DiffusionPreprocessing/DiffPreprocPipeline.sh \ --posData="${PosData}" \ --negData="${NegData}" \ --path="${StudyFolder}" \ --subject="${SubjectID}" \ --echospacing-seconds="${EchoSpacingSec}" \ --PEdir="${PEdir}" \ --gdcoeffs="${Gdcoeffs}" \ --printcom="$PRINTCOM" \ --ensure-even-slices=TRUE \ --extra-eddy-arg=--gpu=TRUE \ --extra-eddy-arg=--nvoxhp=2000 \ --extra-eddy-arg=--ff=10 \ --extra-eddy-arg=--resamp=lsr \ --extra-eddy-arg=--ol_nstd=4 \ --extra-eddy-arg=--peas=TRUE \ --extra-eddy-arg=--cuda-version=10.2 \ --extra-eddy-arg=--detailed_outlier-stats=TRUE \ --extra-eddy-arg=--rms=TRUE \ --extra-eddy-arg=--fwhm=0Note a separate but related concern I have is whether the script is pulling the preprocessed T1w images from the correct location. I suspect it is not, since the data in the study folder variable (e.g.,
StudyFolder="/ZPOOL/data/projects/hcp-ep/imagingcollection01") does not include a folder for preprocessed T1w. I am wondering if this is a separate variable I need to specify, or if I need to replace the unprocessed T1w directory located inStudyFolderwith that containing the preprocessed files. If the latter is the case, is it acceptable to move the whole preprocessed directory, or do I need to move a single file?Kind regards,
Linda
DiffusionPreprocessingBatch.txt