============================================================================== Chromatography - mass spectrometry data import module ============================================================================== The module deposits (GC-MS or LC-MS) data from netCDF data files to hdf5 based database file for subsequent pre-processing aimed to account for various bioanalytical complexities associated with GC-MS/LC-MS technologies Started on Wed, 24 Feb 2021 at 07:40:39 ... Current parameters: datapath = "spectra" dbfilename = "preprocessing_scratch/data.h5" filereadinfo.fileext = ".cdf" filereadinfo.massid = "mass_values" filereadinfo.scanid = "scan_index" filereadinfo.specid = "intensity_values" filereadinfo.timeid = "scan_acquisition_time" filereadinfo.timeunits = "sec" filetype = "netcdf" logfile = "" overwrite_logfile = "no" verbose = 0 5 .cdf files found in spectra Reading netCDF files from 'preprocessing_scratch'... 1. spec-00003.CDF: Successfully deposited -> data.h5 2. spec-00002.CDF: Successfully deposited -> data.h5 3. spec-00000.CDF: Successfully deposited -> data.h5 4. spec-00004.CDF: Successfully deposited -> data.h5 5. spec-00001.CDF: Successfully deposited -> data.h5 Minimal mz: nan Maximal mz: nan Minimal rt (sec): 150.0 Maximal rt (sec): 5580.0 Finished on Wed, 24 Feb 2021 at 07:42:14 in 94.9225549697876 seconds ============================================================================== ============================================================================== Intra/inter-sample mass drift correction module for chromatography - mass spectrometry data ============================================================================== The module is designed to adjust for the inherent variation in instrumental measurements of moelcular m/z ratios between scans (i.e. within sample) Started on Wed, 24 Feb 2021 at 07:42:30 ... Current parameters: dbfilename = "preprocessing_scratch/data.h5" logfile = "" method = "binning" overwrite_logfile = "no" params.binshift = 0.3 params.binsize = 1 params.h5writepath = "sp2D" params.units = "Da" refdbfilename = "" refh5path = "/sp2D" verbose = 0 Traceback (most recent call last): File "/data/ccms-gnps/tools/mshub-gc/release_27/proc/preproc/intrapalign.py", line 281, in <module> params = settings.parameters['params']) File "/data/ccms-gnps/tools/mshub-gc/release_27/proc/preproc/intrapalign.py", line 79, in do_mzalignment mzAlignObj = Binmz(method, params, cmzrange, crtrange) File "/data/ccms-gnps/tools/mshub-gc/release_27/proc/preproc/intrapalign.py", line 117, in __init__ self.__binvals = np.arange(mzrange[0]-binshift,mzrange[1]+binshift+binsize,binsize) ValueError: arange: cannot compute length ============================================================================== Noise filtering module for chromatography - mass spectrometry data ============================================================================== The module is designed to adjust for high frequency noise and baseline distortions of chromatography - mass spectrometry data matrix caused by a variety of instrumental and experimental reasons Started on Wed, 24 Feb 2021 at 07:42:35 ... Current parameters: baselinemethod = "tophat" baselineparams.frame = "50" dbfilename = "preprocessing_scratch/data.h5" logfile = "" overwrite_logfile = "no" params.h5readpath = "sp2D" params.h5writepath = "spproc2D" refdbfilename = "" refh5path = "/sp2D" smoothmethod = "sqfilter" smoothparams.degree = 3 smoothparams.window = "6" verbose = 0 Finished on Wed, 24 Feb 2021 at 07:42:35 in 0.08603882789611816 seconds ============================================================================== ============================================================================== Inter-sample peak alignment module for chromatography - mass spectrometry data ============================================================================== The module is designed to adjust for chromatographic peak position variations at full profile resolution Started on Wed, 24 Feb 2021 at 07:42:39 ... Current parameters: dbfilename = "preprocessing_scratch/data.h5" logfile = "" method = "rspa" overwrite_logfile = "no" params.h5readpath = "spproc2D" params.h5writepath = "spal2D" params.maxpeakshift = "auto" params.minsegwidth = "auto" params.recursion = 1 params.reference = "mean" refdbfilename = "" refh5path = "/sp2D" verbose = 0 Finished on Wed, 24 Feb 2021 at 07:42:39 in 0.06354761123657227 seconds ============================================================================== ============================================================================== Peak detection module for chromatography - mass spectrometry data ============================================================================== The module is designed to identify and integrate chromatographic peaks from generated chromatography - mass spectrometry data matrix Started on Wed, 24 Feb 2021 at 07:42:43 ... Current parameters: dbfilename = "preprocessing_scratch/data.h5" h5readpath = "spal2D" h5writepath = "/spal2D_peakdetect" logfile = "" overwrite_logfile = "no" peak_detect_method = "smoothderiv" peak_detect_params.local_baseline_correction = False peak_detect_params.min_width = "auto" peak_filter_method = "slope" peak_filter_params.global = "no" peak_filter_params.int_thr = "auto" peak_filter_params.left_ang_thr = "auto" peak_filter_params.right_ang_thr = "auto" peak_group_method = "kernel" peak_group_params.frag_pattern = "deconvolution" peak_group_params.individual = "no" peak_group_params.occurence = "common" peak_group_params.rt_tol = "auto" peak_group_params.weighted_density = "no" refdbfilename = "" refh5path = "/sp2D" verbose = 0 Finished on Wed, 24 Feb 2021 at 07:42:43 in 0.06250762939453125 seconds ============================================================================== ============================================================================== Exporting output data module ============================================================================== The module is designed to export results of pre-processing to CSV and TXT formats. Started on Wed, 24 Feb 2021 at 07:42:47 ... Current parameters: dbfilename = "preprocessing_scratch/data.h5" logfile = "" overwrite_logfile = "no" params.export_integral_table = "yes" params.export_metadata_table = "yes" params.export_ms_peak_list = "yes" params.exportpath = "preprocessing_scratch" params.h5fullprofile = "/spal2D" params.h5readpath = "/spal2D_peakdetect" params.output_prefix = "%HDF5_file_name%" params.rt_tolerance = 0.1 params.rts = "*" params.samples = "*" verbose = 0 No datasets found in the h5readpath provided: /spal2D_peakdetect/ ! Finished on Wed, 24 Feb 2021 at 07:42:47 in 0.08532428741455078 seconds ============================================================================== ============================================================================== Exporting output data module ============================================================================== The module is designed to generate HTML report covering results of pre-processing. Started on Wed, 24 Feb 2021 at 07:42:59 ... Current parameters: dbfilename = "preprocessing_scratch/data.h5" logfile = "" overwrite_logfile = "no" params.bottom_plot_height = 400 params.exportpath = "summary_temp" params.h5fullprofile = "/spal2D" params.h5readpath = "/spal2D_peakdetect" params.output_prefix = "gnps-gc" params.plot_width = 1100 params.top_plot_height = 250 verbose = 0 Exporting HTML view from [/scratch/ccms-gnps/651290cac9a04786ac66f7826b2ebe76/preprocess_gcms_data/preprocessing_scratch/data.h5]/spal2D_peakdetect/ to /scratch/ccms-gnps/651290cac9a04786ac66f7826b2ebe76/preprocess_gcms_data/summary_temp/gnps-gc... Error! quantity_integrals not found in [/scratch/ccms-gnps/651290cac9a04786ac66f7826b2ebe76/preprocess_gcms_data/preprocessing_scratch/data.h5]/spal2D_peakdetect/ ! Skipping... Finished on Wed, 24 Feb 2021 at 07:42:59 in 0.0023584365844726562 seconds ============================================================================== tar: summary_temp/gnps-gc: Cannot stat: No such file or directory tar: Exiting with failure status due to previous errors full_CCMS_path 0 saghadejia/GC-MS/MSR-151020a.CDF 1 saghadejia/GC-MS/MSG-131020a.CDF 2 saghadejia/GC-MS/MSB-131020a.CDF 3 saghadejia/GC-MS/MSS-151020a.CDF 4 saghadejia/GC-MS/MSF-151020a.CDF /data/ccms-gnps/tools/miniconda3_gamma/envs/mshub-gc/bin/python /data/ccms-gnps/tools/mshub-gc/release_27/proc/io/importmsdata.py -f netcdf spectra preprocessing_scratch/data.h5 --timeunits 'sec' /data/ccms-gnps/tools/miniconda3_gamma/envs/mshub-gc/bin/python /data/ccms-gnps/tools/mshub-gc/release_27/proc/preproc/intrapalign.py preprocessing_scratch/data.h5 --h5writepath 'sp2D' /data/ccms-gnps/tools/miniconda3_gamma/envs/mshub-gc/bin/python /data/ccms-gnps/tools/mshub-gc/release_27/proc/preproc/noisefilter.py preprocessing_scratch/data.h5 --h5readpath 'sp2D' --h5writepath 'spproc2D' --window 6 --frame 50 /data/ccms-gnps/tools/miniconda3_gamma/envs/mshub-gc/bin/python /data/ccms-gnps/tools/mshub-gc/release_27/proc/preproc/interpalign.py preprocessing_scratch/data.h5 --h5readpath 'spproc2D' --h5writepath 'spal2D' /data/ccms-gnps/tools/miniconda3_gamma/envs/mshub-gc/bin/python /data/ccms-gnps/tools/mshub-gc/release_27/proc/preproc/peakdetect.py preprocessing_scratch/data.h5 --h5readpath 'spal2D' --individual no --frag_pattern deconvolution /data/ccms-gnps/tools/miniconda3_gamma/envs/mshub-gc/bin/python /data/ccms-gnps/tools/mshub-gc/release_27/proc/io/export.py preprocessing_scratch/data.h5 preprocessing_scratch /data/ccms-gnps/tools/miniconda3_gamma/envs/mshub-gc/bin/python /data/ccms-gnps/tools/mshub-gc/release_27/proc/io/report.py --output_prefix gnps-gc preprocessing_scratch/data.h5 summary_temp tar -cvf summary_output/summary.tar summary_temp/gnps-gc Traceback (most recent call last): File "/data/ccms-gnps/tools/mshub-gc/release_27/process_gc.py", line 283, in <module> main() File "/data/ccms-gnps/tools/mshub-gc/release_27/process_gc.py", line 261, in main f = open(output_quant_filename,'r').readlines() FileNotFoundError: [Errno 2] No such file or directory: 'preprocessing_scratch/data_integrals.csv' Tool execution terminates abnormally with exit code [1]