Level 3 model fails due to not finding second levels results #101
Comments
|
I have a preliminary update on this: fitlins/fitlins/interfaces/nistats.py Lines 165 to 180 in c7cc798 during the third level model. The problem is that self.input.contrast_info for a reason I do not yet understand ingests entries from the participants.tsv file and turns them into contrasts. This is how `self.input.contrast_info` looks like for a 3rd level model(Pdb) self.inputs.contrast_info
[{'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_av_rating': 1}], 'type': 't', 'name': 'forrest_av_rating'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_seen_languages': 1}], 'type': 't', 'name': 'forrest_seen_languages'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'hearing_problems_current': 1}], 'type': 't', 'name': 'hearing_problems_current'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_seen_dist': 1}], 'type': 't', 'name': 'forrest_seen_dist'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_av_feeling': 1}], 'type': 't', 'name': 'forrest_av_feeling'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'age': 1}], 'type': 't', 'name': 'age'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_seen_count': 1}], 'type': 't', 'name': 'forrest_seen_count'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_av_storydepth': 1}], 'type': 't', 'name': 'forrest_av_storydepth'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_ad_known': 1}], 'type': 't', 'name': 'forrest_ad_known'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'vision_problems_current': 1}], 'type': 't', 'name': 'vision_problems_current'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_av_fatigue': 1}], 'type': 't', 'name': 'forrest_av_fatigue'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'amplitude_.RIGHT': 1}], 'type': 't', 'name': 'amplitude_.RIGHT'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'handedness': 1}], 'type': 't', 'name': 'handedness'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'hearing_problems_past': 1}], 'type': 't', 'name': 'hearing_problems_past'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_seen': 1}], 'type': 't', 'name': 'forrest_seen'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'vision_problems_past': 1}], 'type': 't', 'name': 'vision_problems_past'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'gender': 1}], 'type': 't', 'name': 'gender'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_av_artist_count': 1}], 'type': 't', 'name': 'forrest_av_artist_count'}]where everything apart from the entry The following fitlins/fitlins/interfaces/nistats.py Line 165 in c7cc798 fitlins/fitlins/interfaces/nistats.py Lines 169 to 172 in c7cc798 input to model.fit(), which results in the failure I observed.
I'm not sure whether the problem lies in the fact that contrast_info is given contrast information that I did not consciously specify anywhere in my model.json (and it is weird that this only happens on third, not second level), or in the fact that it will attempt to give an empty list as input. As for a temporary fix, I modified it like this: for name, weights, type in prepare_contrasts(self.inputs.contrast_info, names):
# Need to add F-test support for intercept (more than one column)
# Currently only taking 0th column as intercept (t-test)
weights = weights[0]
if all(weights == np.zeros(len(names))):
continue
input = (np.array(filtered_files)[weights != 0]).tolist()
design_matrix = pd.DataFrame({'intercept': weights[weights != 0]}) |
|
(I could also be entirely wrong with all of this, but it runs without failure with this added conditional statement) |
|
Sorry, I'm having a lot of trouble getting this to work. I've forgotten... how did we resolve this: |
|
@yarikoptic, do you remember? I've seen this before many times but can't recollect exactly and my computer is a bit disabled due to the model running in the background. Might that have been related to #353 (but thats merged already...)? |
|
Have not read everything carefully, but regarding the automatic inclusion of columns available in (FWIW, this shouldn't only be happening at the 3rd level; it happens at all levels. E.g., if you have |
|
Doesn't |
|
It looks like the issue is with variables created by the |
|
As Adina, isn't it @effigies this fix in pybids? bids-standard/pybids#353, if not then may be something similar, what is the string value there? |
|
@adelavega yes, but that would only make sense for the first level; past that, there's no new model being fit, just contrasts applied to estimates propagated forward. So in the subject-level model, transformations would be needed to explicitly limit what autocontrasts gets applied to. |
|
Thanks for this information! |
|
It definitely shouldn't be happening only at the third level. As @adelavega pointed out, the The mapping, per the BIDS spec, is that session-level analysis automatically pulls in |
'events' |
|
@yarikoptic we ran into the string |
|
I'm guessing that this line may be passing variables in the wrong order, so that |
|
Yeah. That's where I'm looking. It looks like #353 was a partial fix. Though I'm not sure why they can run these models and I can't... |
huh. I actually don't have these files and had no clue I needed them (sorry, must have over-read this when studying the BIDS-spec, but I happily blame Michael for not having them in the first place in the sourcedata). |
|
@effigies would it be helpful if I push my local branch where I committed, merged and cherry-picked all changes we found relevant to my fork here on Github? |
|
You don't need them--they're optional. It sounds like everything is working
as intended (well, at least with respect to this part of things).
…On Wed, Jan 30, 2019, 16:55 Adina Wagner ***@***.*** wrote:
It definitely shouldn't be happening only at the third level. As
@adelavega <https://github.com/adelavega> pointed out, the X field in the
first-level model will implicitly drop any unnamed variables, so that's why
you may not see it happening there. But it should also be happening at
subsequent levels, assuming you have a scans.tsv or sessions.tsv file
containing extra columns. If you don't have anything in those files, then
the behavior you see is exactly as intended.
The mapping, per the BIDS spec, is that session-level analysis
automatically pulls in scans.tsv, subject-level analysis pulls in
sessions.tsv, and dataset-level analysis pulls in participants.tsv.
huh. I actually don't have these files and had no clue I needed them
(sorry, must have over-read this when studying the BIDS-spec, but I happily
blame Michael for not having them in the first place in the sourcedata).
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#101 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AASjPCh7Y2iv-f5dflAV-GFy6kHQ3T0aks5vIiL1gaJpZM4aYK93>
.
|
|
@AdinaWagner Sure, you can go ahead and cherry-pick. I've fixed a couple things, so we're likely to have some clashes, but it would be good to see what you have. |
|
okay, so my current software state is AdinaWagner/pybids branch runinfo-merge and AdinaWagner/fitlins branch BIDSSacc. |
|
I'm also running into the same issue with an image built off current master (ca40343) CrashfileFile: /om/project/voice/bids/scripts/fitlins/crash-20190315-170528-mathiasg-l3_model-2f07306c-9009-48f6-8009-8da1f86d937d.pklz Node: fitlins_wf.l3_model Working directory: /tmp/tmp4nwpaui8/fitlins_wf/l3_model Node inputs: contrast_info = [[{'entities': {'session': '1', 'subject': 'voice969', 'task': 'emosent'}, 'name': 'speech', 'type': 't', 'weights': [{'speech': 1}]}]] Traceback: File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/engine/utils.py", line 99, in nodelist_runner File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py", line 473, in run File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py", line 557, in _run_interface File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py", line 637, in _run_command File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/interfaces/base/core.py", line 375, in run File "/src/fitlins/fitlins/interfaces/nistats.py", line 178, in _run_interface File "/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nistats/second_level_model.py", line 170, in fit ValueError: A second level model requires a list with atleast two first level models or niimgs |
|
@mgxd Can you provide your model? If this is somewhere I can login, it might be easiest to dig through your working directory. |
|
@mathias It looks like you have degenerate inputs. You have two files, that you collapse in layer 2 to one file. Then at layer 3, you only have one file. |
|
Node: nistats
=============
Hierarchy : _l3_model0
Exec ID : _l3_model0
Original Inputs
---------------
* contrast_info : [{'entities': {'session': '1', 'subject': 'voice969', 'task': 'emosent'}, 'name': 'speech', 'type': 't', 'weights': [{'speech': 1}]}]
* stat_files : [['/working/fitlins_wf/l2_model/mapflow/_l2_model0/speech.nii.gz']]
* stat_metadata : [[{'contrast': 'speech', 'session': '1', 'subject': 'voice969', 'suffix': 'stat', 'task': 'emosent'}]] |
(migrating this from pybids where I falsely posted it before) pinging @effigies (@yarikoptic promised me to give you a hydra account that I can finally share the data).
We've stripped our dataset to a bare minimum: 3 subjects, each with 2 runs, and a model file with only one event in X to reduce computation time. You can find the dataset on hydra in the directory
/data/movieloc/backup_store/publish-BIDSSaccon branchslim(please disregard the master branch for now - it is a couple of weeks behind and I don't want to push the currently rather messy state of it).The model (found under
models/movel_v3_smdl_autocon_splitconv_oneevent.jsonis evoked withfitlins . Jan_28 'run' -m models/movel_v3_smdl_autocon_splitconv_oneevent.json --desc highpass --space 'MNI152NLin6Sym' -d $PWD -w 'Jan_28_wd' --n-cpus 3Click here to expand our model
{ "name": "FEF_localizer", "Input": { "session": "movie" }, "Steps": [ { "Level": "run", "Model": { "X": [ "amplitude_.RIGHT" ] }, "Contrasts": [], "AutoContrasts": true, "Transformations": [{ "Name": "Split", "Input": ["amplitude_"], "By": ["trial_type"] }, { "Name": "Convolve", "Input": ["amplitude_.RIGHT"], "Model": "spm" }] }, { "Level": "subject", "AutoContrasts": true }, { "Level": "dataset", "AutoContrasts": true }] }The model fails on level 3 (dataset) with
after producing seemingly sensible output on the first (run) and second (subject) level. As far as I can judge, the files that the level 3 model should ingest have been created.
Here is the traceback
Reproducibility sits in a bar somewhere and laughs its ass of, but I'm trying to also give an overview of custom changes @yarikoptic and I made to the fitlins sourcecode. I don't see any particular relevance of the changes we made to the issue at hand (mostly hardcoding quickfixes for issues that erose), but then again, I'm certainly not the one to judge what is of relevance and what not ;-) , and an attempt to rerun the model would fail without the additional space choice and the selection of only one of two identical bold files being returned.
pybids changes
Do you have any idea what I am missing here to figure out why the dataset level does not work?
The text was updated successfully, but these errors were encountered: