Skip to content

No orientation was found as better than any other #1336

@FelixGersteuer

Description

@FelixGersteuer

During a focus refinement Refine3D job the following error appeared:

ERROR:
No orientation was found as better than any other.

A particle image was compared to the reference and resulted in all-zero
weights (for all orientations). This should not happen, unless your data
has very special characteristics. This has historically happened for some
lower-precision calculations, but multiple fallbacks have since been
implemented. Please report this error to the relion developers at
github.com/3dem/relion/issues


MPI_ABORT was invoked on rank 1 in communicator MPI_COMM_WORLD
with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.

The error appeared once in the beginning of iteration 9. However, I was able to restart the refinement from the previous iteration and it continued to refine until iteration 22. From here I am not able to restart the refinement from any of the previous three iterations.

I would be very grateful for any help from your side!

Environment:

  • OS: Ubuntu 24.04 64bit
  • MPI runtime:
  • RELION version: RELION-5.0.0
  • Memory: 252 GB
  • GPU: 2x RTX 4090

Dataset:

  • Box size: 852 px
  • Pixel size: 0.8336 Å/px
  • Number of particles: 321897
  • Description:

Job options:

  • Type of job: [e.g. Refine3D]
  • Number of MPI processes: 3
  • Number of threads: 8
  • Full command (see note.txt in the job directory):

Error message:

fn_img= 00312431@/home/agwilson/em1/relion_volatile/opticsgroup1_particles.mrcs
op.part_id= 312430
op.min_diff2= 6.80565e+38
group_id= 5
optics_group= 0
ml_model.scale_correction[group_id]= 0.995172
exp_significant_weight= 0
exp_max_weight= -1
ml_model.sigma2_noise[optics_group]=
9e-05 5e-06 3e-06 2e-06 1e-06 1e-06 9e-07 8e-07 7e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 5e-07 5e-07 5e-07 5e-07 4e-07 4e-07 4e-07 3e-07 3e-07 3e-07 3e-07 3e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08

in: /home/agwilson/Software/relion/src/acc/acc_ml_optimiser_impl.h, line 2519
ERROR:
The sum of weights for all orientations was
found to be zero for an image. This should not happen, unless your data
has very special characteristics. Please report this error to
the relion developers at github.com/3dem/relion/issues
in: /home/agwilson/Software/relion/src/acc/acc_ml_optimiser_impl.h, line 2519
ERROR:
ERROR:
The sum of weights for all orientations was
found to be zero for an image. This should not happen, unless your data
has very special characteristics. Please report this error to
the relion developers at github.com/3dem/relion/issues

fn_img= 00313686@/home/agwilson/em1/relion_volatile/opticsgroup1_particles.mrcs
op.part_id= 313685
op.min_diff2= 6.80565e+38
group_id= 15
optics_group= 0
ml_model.scale_correction[group_id]= 1.00625
exp_significant_weight= 0
exp_max_weight= -1
ml_model.sigma2_noise[optics_group]=
9e-05 5e-06 3e-06 2e-06 1e-06 1e-06 9e-07 8e-07 7e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 6e-07 5e-07 5e-07 5e-07 5e-07 4e-07 4e-07 4e-07 3e-07 3e-07 3e-07 3e-07 3e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 2e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 1e-07 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 9e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 8e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08 7e-08

in: /home/agwilson/Software/relion/src/acc/acc_ml_optimiser_impl.h, line 2519
ERROR:
The sum of weights for all orientations was
found to be zero for an image. This should not happen, unless your data
has very special characteristics. Please report this error to
the relion developers at github.com/3dem/relion/issues
in: /home/agwilson/Software/relion/src/acc/acc_ml_optimiser_impl.h, line 2519
ERROR:
ERROR:
The sum of weights for all orientations was
found to be zero for an image. This should not happen, unless your data
has very special characteristics. Please report this error to
the relion developers at github.com/3dem/relion/issues
follower 1 encountered error: === Backtrace ===
/home/agwilson/Software/relion/build/bin//relion_refine_mpi(_ZN11RelionErrorC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_l+0x77) [0x58163de388e7]
/home/agwilson/Software/relion/build/bin//relion_refine_mpi(+0xbe4e3) [0x58163de114e3]
/home/agwilson/Software/relion/build/bin//relion_refine_mpi(+0x34205d) [0x58163e09505d]
/lib/x86_64-linux-gnu/libgomp.so.1(+0x2279f) [0x73d20f6ab79f]
/lib/x86_64-linux-gnu/libc.so.6(+0x9caa4) [0x73d20e89caa4]
/lib/x86_64-linux-gnu/libc.so.6(+0x129c6c) [0x73d20e929c6c]

ERROR:
The sum of weights for all orientations was
found to be zero for an image. This should not happen, unless your data
has very special characteristics. Please report this error to
the relion developers at github.com/3dem/relion/issues

MPI_ABORT was invoked on rank 1 in communicator MPI_COMM_WORLD
with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions