Hello,
We are slowly having a look at v3. In initial tests we run into some issues importing v2 projects, however. To what extent should this be possible? Should it just work, or is some editing necessary?
I have now tried several different projects and have yet to find one that does not lead to an error (and even worse: crash).
For example:
==> Loading CCPN project `mp44_exon1_final_PW20170220c`
Error loading file for: <ccp.nmr.Nmr.NmrProject [`template`]>
Reading: <_io.TextIOWrapper name=`/Users/patrick/Dropbox/Work/Publications/2016/Hoop_PNAS_PolyQ_core/Data/Exon1_MP44_double_label_PBSZF_HFCN_150211/mp44_exon1_final_PW20170220c.ccpn/ccpnv3/ccp/nmr/Nmr/template+template_patrick_2012-03-13-10-18-59-685_00001.xml` mode=`r` encoding=`UTF-8`>
Last xml tag read: NMR.Experiment.labeledMixtures
Parser state was: reading
Current object was: <ccp.nmr.Nmr.Experiment [`template`, 1]>
Traceback (most recent call last):
File `/Users/patrick/software/ccpnmr/miniconda/lib/python3.5/runpy.py`, line 193, in _run_module_as_main
`__main__`, mod_spec)
File `/Users/patrick/software/ccpnmr/miniconda/lib/python3.5/runpy.py`, line 85, in _run_code
exec(code, run_globals)
File `/Users/patrick/software/ccpnmr/src/python/ccpn/AnalysisAssign/__main__.py`, line 43, in <module>
application.start()
File `/Users/patrick/software/ccpnmr/src/python/ccpn/framework/Framework.py`, line 354, in start
project = self.loadProject(projectPath)
File `/Users/patrick/software/ccpnmr/src/python/ccpn/framework/Framework.py`, line 1418, in loadProject
project = coreIo.loadProject(path, useFileLogger=self.useFileLogger, level=self.level)
File `/Users/patrick/software/ccpnmr/src/python/ccpn/core/_implementation/Io.py`, line 41, in loadProject
project = _loadNmrProject(path, useFileLogger=useFileLogger)
File `/Users/patrick/software/ccpnmr/src/python/ccpn/core/_implementation/Io.py`, line 77, in _loadNmrProject
apiProject = apiIo.loadProject(path, useFileLogger=useFileLogger)
File `/Users/patrick/software/ccpnmr/src/python/ccpnmodel/ccpncore/lib/Io/Api.py`, line 473, in loadProject
_compressDataLocations(project)
File `/Users/patrick/software/ccpnmr/src/python/ccpnmodel/ccpncore/lib/Io/Api.py`, line 1323, in _compressDataLocations
dataStore.repointToDataUrl(targetUrl)
File `/Users/patrick/software/ccpnmr/src/python/ccpnmodel/ccpncore/lib/_ccp/general/DataLocation/AbstractDataStore.py`, line 93, in repointToDataUrl
for topObject in set(x.topObject for x in self.nmrDataSources):
File `/Users/patrick/software/ccpnmr/src/python/ccpnmodel/ccpncore/api/ccp/general/DataLocation.py`, line 5964, in getNmrDataSources
tobj.load()
File `/Users/patrick/software/ccpnmr/src/python/ccpnmodel/ccpncore/api/memops/Implementation.py`, line 21141, in load
self.loadFrom(repositories[0])
File `/Users/patrick/software/ccpnmr/src/python/ccpnmodel/ccpncore/api/memops/Implementation.py`, line 21175, in loadFrom
XmlIO.loadTopObject(repository.url.getDataLocation(), self)
File `/Users/patrick/software/ccpnmr/src/python/ccpnmodel/ccpncore/memops/format/xml/XmlIO.py`, line 242, in loadTopObject
return loadFromStream(stream, topObjId=topObjId, topObject=topObject)
File `/Users/patrick/software/ccpnmr/src/python/ccpnmodel/ccpncore/memops/format/xml/XmlIO.py`, line 281, in loadFromStream
partialLoad=partialLoad)
File `/Users/patrick/software/ccpnmr/src/python/ccpnmodel/ccpncore/xml/memops/Implementation.py`, line 4264, in loadFromStream
% ((curMap[`content`][`.qName`], clazz.__name__, vv[1:-1]),))
ccpnmodel.ccpncore.memops.ApiError.ApiError: No object found with package,class,key: (`ccp.molecule.LabeledMolecule`, `LabeledMixture`, [1])
Any thoughts on how to proceed? These are all ssNMR projects, in case that matters.
Best wishes,
Patrick
Hi Patrick,
in principle V2 import should work seamlessly and V2 projects should just open in V3.
Looking at your error, I wonder if these are projects in which you have specified particular labelling for your samples?
At the moment V3 doesn`t support sample labelling. We are still unsure whether we will ever do so - our impression is that the labelling in V2 isn`t used very much. We can perhaps be persuaded otherwise, if users feel this is an integral part to how they use Analysis, but so far there are other things which seem to be rather higher up on people`s wish lists.
But obviously you should still be able to read in a V2 project with labelling and we need to make sure that the whole labelling stuff is just skipped and the program doesn`t crash!
Best wishes,
Vicky
Hi Vicky,
Yes, in this case we used the labeling features. I quite liked them, but perhaps it did not (yet) offer concrete functionality. I thought there was an idea to use the labeling information for (automated/facilitated) assignment of differently labeled proteins in ssNMR. For example, there is this nice paper: URL: link.springer.com/article/10.1007/s10858-009-9338-7 that I always think of in this context.
How soon can you expect to have the import feature deal with this issue? Could you have the import process pop up a notification that this was detected, but not transferred? Then people would know to take this into account.
I can also add that we also tried some other projects that did not use the labeling feature. We`ll have to see what errors those are.
Thanks
Patrick
PS. Why do you not allow URLs in this forum? Is it really necessary to disallow those?
Hi Patrick,
URLs are disabled for the first two posts for new users. We had to do for avoiding spammers.
Hi all
I am slightly confused as to why this doesn’t work, as LabeledMolecule is part of the api ands so should still be there?
Regards
Gary
Let me know if we can share project files/data to allow testing.
Patrick
I have previously reported this issue would like to elevate it.
It seems to be a part of the data model that the V2 to V3 conversion code does not handle and means that all past projects where labelling info was included (i.e. any where the ECI was used to prepare the project for deposition, at least) can not be imported into V3.
(03-18-2020, 04:54 pm)bosmith Wrote: I have previously reported this issue would like to elevate it.
It seems to be a part of the data model that the V2 to V3 conversion code does not handle and means that all past projects where labelling info was included (i.e. any where the ECI was used to prepare the project for deposition, at least) can not be imported into V3.
Yikes. I think we have such information in many of our projects from version 2.... hopefully this can be addressed soon.
Hi Brian,
sorry - I think this had slipped through our net from when you previously raised this. Anyway, I think Ed is going to look into this as soon as we have the 2.5 and 3.0.1 versions sorted and out. I feel like I think we`ll be done with these nearly every day and then each time some issue crops up and turns out to be harder to fix than hoped...
Vicky
Hello Vicky,
I see that v2.5 and 3.01 are out. Excellent! Thanks!
Does that mean you are revisiting the issues with importing into v3? The ability to import our v2 projects would greatly facilitate our attempts to at least try out v3.. Let me know if we can share test project files to resolve these issues.
Best wishes,
patrick
Hi Patrick,
yes, we are. At the moment we are improving the NEF import in V3, so that you can import only part of a NEF file which will be crucial for importing assignments/peak lists etc. from FLYA/CYANA/ARIA/XPLOR. Then we`ll sort out the issue of importing V2 projects with labelling patterns - though the labelling patterns will be ignored in V3 for the time being. We`re still trying to work out how much this feature is actually used and whether it is worth implementing in V3. This is probably a feature used most by the ssNMR community, though I know several groups don`t bother with it. Would be interesting to know how much you make use of it (beyond just giving your samples a labelling scheme).
Best wishes,
Vicky
(05-06-2020, 07:42 am)VickyH Wrote: Hi Patrick,
yes, we are. At the moment we are improving the NEF import in V3, so that you can import only part of a NEF file which will be crucial for importing assignments/peak lists etc. from FLYA/CYANA/ARIA/XPLOR. Then we`ll sort out the issue of importing V2 projects with labelling patterns - though the labelling patterns will be ignored in V3 for the time being. We`re still trying to work out how much this feature is actually used and whether it is worth implementing in V3. This is probably a feature used most by the ssNMR community, though I know several groups don`t bother with it. Would be interesting to know how much you make use of it (beyond just giving your samples a labelling scheme).
Best wishes,
Vicky
Hello Vicky,
I don`t think we ever did much with the labeling schemes in CCPN, except keeping tracking/organizing the sample types. I recall that at the time the CCPN team seemed to suggest future incorporation into more automated ssNMR analysis tools/methods. Although that seems to make sense to me, it never seems to have materialised (and I am not aware of ongoing work toward this at this moment either)
Right now (as mentioned before), we mostly just need a way to use existing data in v3 to make that program useful to us.
Patrick
Hi Patrick and Brian,
we have finally got a fix for importing V2 projects containing labelling schemes. It turns out that the problem was caused by a change in the data model. So we won`t be able to fix this fully until we do a new release. But in the mean time we have a little plugin which will allow you to open your projects in V3.
Make sure you are up-to-date with all the V3.0.1 updates.
Instructions are contained in a video here: https://youtu.be/Cz-DwGuAurY
Basically you will need to go to Plugins / User Plugins / Filter Labelled Mixtures, select your V2 project, click on Run and then after that you will be able to drag and drop the project into V3 as normal. The plugin will edit your xml files. It does keep a copy of the original file, but you may prefer to do all of this on a copy of your V2 project, rather than the original one (especially if you still want to be able to open the original one in V2).
Best wishes,
Vicky