Radium

Posted: **29 May 2020 09:13**

Seems like VST3 is supported on Linux in JUCE 6 (https://forum.juce.com/t/vst3-support-on-linux/31872/8). But JUCE 6 has not been officially released yet. JUCE 6 is announced to be released in June though: https://juce.com/discover/stories/announcing-juce-6

Posted: **29 May 2020 09:39**

Can you run the following command in a terminal when Redux freezes? (make sure only one radium is running)

echo "thread apply all bt" | gdb attach `pidof radium_linux.bin`

And post the output here. This should narrow down where redux or radium is freezing. It might even tell us exactly where it's freezing.

Or, run

echo "thread apply all bt" | gdb attach `pidof radium_linux.bin` >radium_bt.txt

And attach the file "radium_bt.txt" here.

Posted: **29 May 2020 13:37**

Also, from the log it seems like you open the GUI of the first redux instance before loading the second redux instance. Can you confirm that this recipe causes a freeze? (i.e. no opening of a GUI or anything else)

1. Start Radium
2. Create first redux instance
3. Create second redux instance
4. Freeze

Posted: **29 May 2020 15:03**

kjetil wrote: ↑29 May 2020 13:37 Also, from the log it seems like you open the GUI of the first redux instance before loading the second redux instance. Can you confirm that this recipe causes a freeze? (i.e. no opening of a GUI or anything else)

1. Start Radium
2. Create first redux instance
3. Create second redux instance
4. Freeze

I can confirm this above. It doesn't matter if I open the first gui or not, it freezes on both.

kjetil wrote: ↑29 May 2020 09:39 Can you run the following command in a terminal when Redux freezes? (make sure only one radium is running)

echo "thread apply all bt" | gdb attach `pidof radium_linux.bin`
And post the output here. This should narrow down where redux or radium is freezing. It might even tell us exactly where it's freezing.

Or, run

echo "thread apply all bt" | gdb attach `pidof radium_linux.bin` >radium_bt.txt
And attach the file "radium_bt.txt" here.

So, you're not going to believe this: I was able to open 2 reduxes today, but 3 and radium froze.

Code: Select all

[brandon@brandon-thinkpad ~]$ echo "thread apply all bt" | sudo gdb attach `pidof radium_linux.bin` 
GNU gdb (GDB) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
attach: No such file or directory.
Attaching to process 11769
[New LWP 11770]
[New LWP 11771]
[New LWP 11772]
[New LWP 11773]
[New LWP 11774]
[New LWP 11775]
[New LWP 11776]
[New LWP 11777]
[New LWP 11778]
[New LWP 11779]
[New LWP 11780]
[New LWP 11791]
[New LWP 11792]
[New LWP 11803]
[New LWP 11804]
[New LWP 11807]
[New LWP 11808]
[New LWP 11809]
[New LWP 11810]
[New LWP 11811]
[New LWP 11812]
[New LWP 11813]
[New LWP 11814]
[New LWP 11815]
[New LWP 11816]
[New LWP 11819]
[New LWP 11820]
[New LWP 11831]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
--Type <RET> for more, q to quit, c to continue without paging--0x00007f628af236bf in __lll_clocklock_wait () from /usr/lib/libpthread.so.0
(gdb) quit
A debugging session is active.

	Inferior 1 [process 11769] will be detached.

Quit anyway? (y or n) [answered Y; input not from terminal]
Detaching from program: /opt/radium/radium_linux.bin, process 11769
[Inferior 1 (process 11769) detached]

I had to run gdb as root, as it gave an error of operation not permitted.

And, just to confirm, running the nvidia driver would affect stability? I am on an optimus laptop, so I might try running Radium on the intel driver.

And as far as running binary plugins on non-standard distros, I actually didn't know that this mattered, but that is lame. If it is going to be a pain to use vsts, I might try to transition away from using them if that is the case (faust and pd rock). I don't have these kinds of issues with Renoise though. Artix shouldn't be much different from Arch, should it? Artix even uses the standard Arch community repos for a lot of the audio stuff.

EDIT: Thanks for the build instructions for Surge, I will definitely try that. I like Surge a lot. I wasn't using the newest Arch community vst3 version since it didn't seem that Radium supported loading vst3, but looking forward to seeing how that turns out around June, like Kjetil said.

Posted: **29 May 2020 15:11**

Seems like your gdb didn't accept the "attach" command, so it only printed an error message. Can you try this instead?

Code: Select all

cd /opt/radium
echo "thread apply all bt" | gdb ./radium_linux.bin `pidof radium_linux.bin`

Posted: **29 May 2020 15:15**

And, just to confirm, running the nvidia driver would affect stability? I am on an optimus laptop, so I might try running Radium on the intel driver.

The nvidia driver is the only one I've not had trouble with. Can't remember any crash reports caused by the nvidia driver either.

Posted: **29 May 2020 15:19**

I don't have these kinds of issues with Renoise though.

Hmm, on Renoise you can run plugins in a sandbox. Then the host won't die if the plugin dies. (uses a bit more resources if you do this). Another thing is that Renoise doesn't use OpenGL, as far as I know, so you might have less trouble with plugins using OpenGL, but that's just speculation.

Posted: **29 May 2020 15:28**

kjetil wrote: ↑29 May 2020 15:11 Seems like your gdb didn't accept the "attach" command, so it only printed an error message. Can you try this instead?
Code: Select all
cd /opt/radium
echo "thread apply all bt" | gdb ./radium_linux.bin `pidof radium_linux.bin`

And if that didn't give more information either (you should see hundreds of lines), then can you try this?

Code: Select all

cd /opt/radium
gdb ./radium_linux.bin `pidof radium_linux.bin`
#(wait for gdb to start)
thread apply all bt

Posted: **29 May 2020 16:10**

Here we are: (had to run as root for gdb to be allowed)

Code: Select all

[brandon@brandon-thinkpad radium]$ sudo gdb ./radium_linux.bin `pidof radium_linux.bin`
[sudo] password for brandon: 
GNU gdb (GDB) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./radium_linux.bin...
Attaching to program: /opt/radium/radium_linux.bin, process 16065
[New LWP 16066]
[New LWP 16067]
[New LWP 16068]
[New LWP 16069]
[New LWP 16070]
[New LWP 16071]
[New LWP 16072]
[New LWP 16073]
[New LWP 16074]
[New LWP 16075]
[New LWP 16076]
[New LWP 16087]
[New LWP 16088]
[New LWP 16099]
[New LWP 16100]
[New LWP 16103]
[New LWP 16104]
[New LWP 16105]
[New LWP 16106]
[New LWP 16107]
[New LWP 16108]
[New LWP 16109]
[New LWP 16110]
[New LWP 16111]
[New LWP 16112]
[New LWP 16115]
[New LWP 16116]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
--Type <RET> for more, q to quit, c to continue without paging--
0x00007fcc816b36bf in __lll_clocklock_wait () from /usr/lib/libpthread.so.0
(gdb) thread apply all bt

Thread 28 (Thread 0x7fcbc3fff700 (LWP 16116)):
#0  0x00007fcc816b28f4 in do_futex_wait.constprop () at /usr/lib/libpthread.so.0
#1  0x00007fcc816b29f8 in __new_sem_wait_slow.constprop.0 () at /usr/lib/libpthread.so.0
#2  0x000056020e1212e6 in cpp11onmulticore::Semaphore::wait() (this=<optimized out>) at OpenGL/../common/sema.h:156
#3  radium::Semaphore::wait() (this=0x5602101f0878 <t1_to_t2_queue+56>) at OpenGL/../common/Semaphores.hpp:79
#4  radium::BaseQueueStack<T1_data*>::wait() (this=0x5602101f0840 <t1_to_t2_queue>) at OpenGL/../common/QueueStack.hpp:83
#5  radium::BaseQueueStack<T1_data*>::get() (this=0x5602101f0840 <t1_to_t2_queue>) at OpenGL/../common/QueueStack.hpp:105
#6  T2_thread_func (offscreen_context=0x56021c3bd2d0, offscreen=<optimized out>) at OpenGL/T2.cpp:197
#7  (anonymous namespace)::T2_Thread::run (this=0x560210e3ec20) at OpenGL/T2.cpp:275
#8  (anonymous namespace)::T2_Thread::run() (this=0x560210e3ec20) at OpenGL/T2.cpp:272
#9  0x00007fcc856cedbf in  () at /usr/lib/libQt5Core.so.5
#10 0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#11 0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 27 (Thread 0x7fcbc8eca700 (LWP 16115)):
#0  0x00007fcc815ce05f in poll () at /usr/lib/libc.so.6
#1  0x00007fcc691a1774 in  () at /usr/lib/libnvidia-glcore.so.440.82
--Type <RET> for more, q to quit, c to continue without paging--
#2  0x00007fcc69289a25 in  () at /usr/lib/libnvidia-glcore.so.440.82
#3  0x00007fcc691800ce in  () at /usr/lib/libnvidia-glcore.so.440.82
#4  0x00007fcc71e4e1d4 in  () at /usr/lib/libGLX_nvidia.so.0
#5  0x00007fcc71e23071 in glXSwapBuffers () at /usr/lib/libGLX_nvidia.so.0
#6  0x00007fcc72c106a9 in  () at /usr/lib/qt/plugins/xcbglintegrations/libqxcb-glx-integration.so
#7  0x000056020e12fd17 in MyQtThreadedWidget::swap() (this=0x56021baae880) at OpenGL/Widget.cpp:1506
#8  0x000056020e1343dd in MyQtThreadedWidget::updateEvent() (this=0x56021baae880) at OpenGL/Widget.cpp:1974
#9  0x000056020e1354a0 in vl::OpenGLContext::dispatchRunEvent() (this=0x56021c3bbf00) at bin/packages/Visualization-Library-master/src/vlGraphics/OpenGLContext.hpp:403
#10 vlQt5::Qt5ThreadedWidget::MyThread::run() (this=0x56021c3bbef0) at bin/packages/Visualization-Library-master/src/vlQt5/Qt5ThreadedWidget.hpp:220
#11 0x00007fcc856cedbf in  () at /usr/lib/libQt5Core.so.5
#12 0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#13 0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 26 (Thread 0x7fcbf9ffb700 (LWP 16112)):
#0  0x00007fcc815ce05f in poll () at /usr/lib/libc.so.6
#1  0x000056020e80b3f5 in juce::(anonymous namespace)::AlsaClient::MidiInputThread::run() (this=0x56021390c590) at ../../JuceLibraryCode/modules/juce_audio_devices/native/juce_linux_Midi.cpp:337
#2  0x000056020e8edd6f in juce::Thread::threadEntryPoint() (this=0x56021390c590) at ../../JuceLibraryCode/modules/juce_core/threads/juce_Thread.cpp:96
#3  0x000056020e8edfa9 in juce::juce_threadEntryPoint(void*) (userData=<optimized out>) at ../../JuceLibraryCode/modules/juce_core/threads/juce_Thread.cpp:118
--Type <RET> for more, q to quit, c to continue without paging--c
#4  juce::threadEntryProc(void*) (userData=<optimized out>) at ../../JuceLibraryCode/modules/juce_core/native/juce_posix_SharedCode.h:834
#5  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#6  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 25 (Thread 0x7fcbfa7fc700 (LWP 16111)):
#0  0x00007fcc816b28f4 in do_futex_wait.constprop () at /usr/lib/libpthread.so.0
#1  0x00007fcc816b29f8 in __new_sem_wait_slow.constprop.0 () at /usr/lib/libpthread.so.0
#2  0x000056020e066226 in cpp11onmulticore::Semaphore::wait() (this=<optimized out>) at audio/../common/sema.h:156
#3  radium::Semaphore::wait() (this=0x5602100e85f8 <soundproducer_queue+56>) at audio/../common/Semaphores.hpp:79
#4  radium::BaseQueueStack<SoundProducer*>::wait() (this=0x5602100e85c0 <soundproducer_queue>) at audio/../common/QueueStack.hpp:83
#5  radium::BaseQueueStack<SoundProducer*>::get() (this=0x5602100e85c0 <soundproducer_queue>) at audio/../common/QueueStack.hpp:105
#6  (anonymous namespace)::Runner::run() (this=0x560218a2cd50) at audio/MultiCore.cpp:311
#7  0x00007fcc856cedbf in  () at /usr/lib/libQt5Core.so.5
#8  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#9  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 24 (Thread 0x7fcbfaffd700 (LWP 16110)):
#0  0x00007fcc816b28f4 in do_futex_wait.constprop () at /usr/lib/libpthread.so.0
#1  0x00007fcc816b29f8 in __new_sem_wait_slow.constprop.0 () at /usr/lib/libpthread.so.0
#2  0x000056020e066226 in cpp11onmulticore::Semaphore::wait() (this=<optimized out>) at audio/../common/sema.h:156
#3  radium::Semaphore::wait() (this=0x5602100e85f8 <soundproducer_queue+56>) at audio/../common/Semaphores.hpp:79
#4  radium::BaseQueueStack<SoundProducer*>::wait() (this=0x5602100e85c0 <soundproducer_queue>) at audio/../common/QueueStack.hpp:83
#5  radium::BaseQueueStack<SoundProducer*>::get() (this=0x5602100e85c0 <soundproducer_queue>) at audio/../common/QueueStack.hpp:105
#6  (anonymous namespace)::Runner::run() (this=0x560218a1b540) at audio/MultiCore.cpp:311
#7  0x00007fcc856cedbf in  () at /usr/lib/libQt5Core.so.5
#8  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#9  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 23 (Thread 0x7fcbfb7fe700 (LWP 16109)):
#0  0x00007fcc816b28f4 in do_futex_wait.constprop () at /usr/lib/libpthread.so.0
#1  0x00007fcc816b29f8 in __new_sem_wait_slow.constprop.0 () at /usr/lib/libpthread.so.0
#2  0x000056020e066226 in cpp11onmulticore::Semaphore::wait() (this=<optimized out>) at audio/../common/sema.h:156
#3  radium::Semaphore::wait() (this=0x5602100e85f8 <soundproducer_queue+56>) at audio/../common/Semaphores.hpp:79
#4  radium::BaseQueueStack<SoundProducer*>::wait() (this=0x5602100e85c0 <soundproducer_queue>) at audio/../common/QueueStack.hpp:83
#5  radium::BaseQueueStack<SoundProducer*>::get() (this=0x5602100e85c0 <soundproducer_queue>) at audio/../common/QueueStack.hpp:105
#6  (anonymous namespace)::Runner::run() (this=0x560218a25680) at audio/MultiCore.cpp:311
#7  0x00007fcc856cedbf in  () at /usr/lib/libQt5Core.so.5
#8  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#9  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 22 (Thread 0x7fcbfbfff700 (LWP 16108)):
#0  0x00007fcc816b28f4 in do_futex_wait.constprop () at /usr/lib/libpthread.so.0
#1  0x00007fcc816b29f8 in __new_sem_wait_slow.constprop.0 () at /usr/lib/libpthread.so.0
#2  0x000056020e066226 in cpp11onmulticore::Semaphore::wait() (this=<optimized out>) at audio/../common/sema.h:156
#3  radium::Semaphore::wait() (this=0x5602100e85f8 <soundproducer_queue+56>) at audio/../common/Semaphores.hpp:79
#4  radium::BaseQueueStack<SoundProducer*>::wait() (this=0x5602100e85c0 <soundproducer_queue>) at audio/../common/QueueStack.hpp:83
#5  radium::BaseQueueStack<SoundProducer*>::get() (this=0x5602100e85c0 <soundproducer_queue>) at audio/../common/QueueStack.hpp:105
#6  (anonymous namespace)::Runner::run() (this=0x5602189e24e0) at audio/MultiCore.cpp:311
#7  0x00007fcc856cedbf in  () at /usr/lib/libQt5Core.so.5
#8  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#9  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 21 (Thread 0x7fcc0ca9b700 (LWP 16107)):
#0  0x00007fcc816b28f4 in do_futex_wait.constprop () at /usr/lib/libpthread.so.0
#1  0x00007fcc816b29f8 in __new_sem_wait_slow.constprop.0 () at /usr/lib/libpthread.so.0
#2  0x000056020e066226 in cpp11onmulticore::Semaphore::wait() (this=<optimized out>) at audio/../common/sema.h:156
#3  radium::Semaphore::wait() (this=0x5602100e85f8 <soundproducer_queue+56>) at audio/../common/Semaphores.hpp:79
#4  radium::BaseQueueStack<SoundProducer*>::wait() (this=0x5602100e85c0 <soundproducer_queue>) at audio/../common/QueueStack.hpp:83
#5  radium::BaseQueueStack<SoundProducer*>::get() (this=0x5602100e85c0 <soundproducer_queue>) at audio/../common/QueueStack.hpp:105
#6  (anonymous namespace)::Runner::run() (this=0x560218928c00) at audio/MultiCore.cpp:311
#7  0x00007fcc856cedbf in  () at /usr/lib/libQt5Core.so.5
#8  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#9  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 20 (Thread 0x7fcc0d29c700 (LWP 16106)):
#0  0x00007fcc816b28f4 in do_futex_wait.constprop () at /usr/lib/libpthread.so.0
#1  0x00007fcc816b29f8 in __new_sem_wait_slow.constprop.0 () at /usr/lib/libpthread.so.0
#2  0x000056020e066226 in cpp11onmulticore::Semaphore::wait() (this=<optimized out>) at audio/../common/sema.h:156
#3  radium::Semaphore::wait() (this=0x5602100e85f8 <soundproducer_queue+56>) at audio/../common/Semaphores.hpp:79
#4  radium::BaseQueueStack<SoundProducer*>::wait() (this=0x5602100e85c0 <soundproducer_queue>) at audio/../common/QueueStack.hpp:83
#5  radium::BaseQueueStack<SoundProducer*>::get() (this=0x5602100e85c0 <soundproducer_queue>) at audio/../common/QueueStack.hpp:105
#6  (anonymous namespace)::Runner::run() (this=0x560213ae8b60) at audio/MultiCore.cpp:311
#7  0x00007fcc856cedbf in  () at /usr/lib/libQt5Core.so.5
#8  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#9  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 19 (Thread 0x7fcc0da9d700 (LWP 16105)):
#0  0x00007fcc816b28f4 in do_futex_wait.constprop () at /usr/lib/libpthread.so.0
#1  0x00007fcc816b29f8 in __new_sem_wait_slow.constprop.0 () at /usr/lib/libpthread.so.0
#2  0x000056020e066226 in cpp11onmulticore::Semaphore::wait() (this=<optimized out>) at audio/../common/sema.h:156
#3  radium::Semaphore::wait() (this=0x5602100e85f8 <soundproducer_queue+56>) at audio/../common/Semaphores.hpp:79
#4  radium::BaseQueueStack<SoundProducer*>::wait() (this=0x5602100e85c0 <soundproducer_queue>) at audio/../common/QueueStack.hpp:83
#5  radium::BaseQueueStack<SoundProducer*>::get() (this=0x5602100e85c0 <soundproducer_queue>) at audio/../common/QueueStack.hpp:105
#6  (anonymous namespace)::Runner::run() (this=0x5602189d9850) at audio/MultiCore.cpp:311
#7  0x00007fcc856cedbf in  () at /usr/lib/libQt5Core.so.5
#8  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#9  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 18 (Thread 0x7fcc0f7fe700 (LWP 16104)):
#0  0x00007fcc815ce05f in poll () at /usr/lib/libc.so.6
#1  0x00007fcc8557af38 in  () at /usr/lib/libglib-2.0.so.0
#2  0x00007fcc8552bc03 in g_main_loop_run () at /usr/lib/libglib-2.0.so.0
#3  0x00007fcc7fc921a8 in  () at /usr/lib/libgio-2.0.so.0
#4  0x00007fcc85555511 in  () at /usr/lib/libglib-2.0.so.0
#5  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#6  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 17 (Thread 0x7fcc0ffff700 (LWP 16103)):
#0  0x00007fcc815ce05f in poll () at /usr/lib/libc.so.6
#1  0x00007fcc8557af38 in  () at /usr/lib/libglib-2.0.so.0
#2  0x00007fcc8552c221 in g_main_context_iteration () at /usr/lib/libglib-2.0.so.0
#3  0x00007fcc8552c272 in  () at /usr/lib/libglib-2.0.so.0
#4  0x00007fcc85555511 in  () at /usr/lib/libglib-2.0.so.0
#5  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#6  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 16 (Thread 0x7fcc1d5a6700 (LWP 16100)):
#0  0x00007fcc816afe32 in pthread_cond_wait@@GLIBC_2.3.2 () at /usr/lib/libpthread.so.0
#1  0x00007fcc85338060 in  () at /usr/lib/libjack.so.0
#2  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#3  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 15 (Thread 0x7fcc1dda7700 (LWP 16099)):
#0  0x00007fcc816b28f4 in do_futex_wait.constprop () at /usr/lib/libpthread.so.0
#1  0x00007fcc816b29f8 in __new_sem_wait_slow.constprop.0 () at /usr/lib/libpthread.so.0
#2  0x000056020e0adf46 in cpp11onmulticore::Semaphore::wait() (this=<optimized out>) at audio/../common/sema.h:156
#3  radium::Semaphore::wait() (this=0x7fcc1e5aa048) at audio/../common/Semaphores.hpp:79
#4  radium::BaseQueueStack<(anonymous namespace)::RecordingSlice*>::wait (this=0x7fcc1e5aa010) at audio/../common/QueueStack.hpp:83
#5  radium::BaseQueueStack<(anonymous namespace)::RecordingSlice*>::get (this=0x7fcc1e5aa010) at audio/../common/QueueStack.hpp:105
#6  (anonymous namespace)::SampleRecorderThread::run() (this=<optimized out>) at audio/SampleRecorder.cpp:466
#7  0x00007fcc856cedbf in  () at /usr/lib/libQt5Core.so.5
#8  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#9  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 14 (Thread 0x7fcc1f1ab700 (LWP 16088)):
#0  0x00007fcc816b28f4 in do_futex_wait.constprop () at /usr/lib/libpthread.so.0
#1  0x00007fcc816b29f8 in __new_sem_wait_slow.constprop.0 () at /usr/lib/libpthread.so.0
#2  0x000056020df829b6 in cpp11onmulticore::Semaphore::wait() (this=<optimized out>) at midi/../common/sema.h:156
#3  radium::Semaphore::wait() (this=0x560211bb0c08) at midi/../common/Semaphores.hpp:79
#4  radium::BaseQueueStack<_midi_event_t>::wait() (this=0x560211bb0bd0) at midi/../common/QueueStack.hpp:83
#5  radium::BaseQueueStack<_midi_event_t>::get() (this=0x560211bb0bd0) at midi/../common/QueueStack.hpp:105
#6  (anonymous namespace)::RecordingQueuePullThread::run() (this=<optimized out>) at midi/midi_i_input.cpp:332
#7  0x00007fcc856cedbf in  () at /usr/lib/libQt5Core.so.5
#8  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#9  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 13 (Thread 0x7fcc1f9ac700 (LWP 16087)):
#0  0x00007fcc815ce05f in poll () at /usr/lib/libc.so.6
#1  0x00007fcc8557af38 in  () at /usr/lib/libglib-2.0.so.0
#2  0x00007fcc8552c221 in g_main_context_iteration () at /usr/lib/libglib-2.0.so.0
#3  0x00007fcc859050f1 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () at /usr/lib/libQt5Core.so.5
#4  0x00007fcc858acbfc in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () at /usr/lib/libQt5Core.so.5
#5  0x00007fcc856cdbf2 in QThread::exec() () at /usr/lib/libQt5Core.so.5
#6  0x00007fcc73e59098 in  () at /usr/lib/libQt5DBus.so.5
#7  0x00007fcc856cedbf in  () at /usr/lib/libQt5Core.so.5
#8  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#9  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 12 (Thread 0x7fcc734f3700 (LWP 16076)):
#0  0x00007fcc816afe32 in pthread_cond_wait@@GLIBC_2.3.2 () at /usr/lib/libpthread.so.0
#1  0x00007fcc818ecc01 in __gthread_cond_wait (__mutex=<optimized out>, __cond=<optimized out>) at /build/gcc/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:865
#2  std::condition_variable::wait(std::unique_lock<std::mutex>&) (this=<optimized out>, __lock=...) at /build/gcc/src/gcc/libstdc++-v3/src/c++11/condition_variable.cc:53
#3  0x00007fcc84091d2b in  () at /usr/lib/libQt5WebKit.so.5
#4  0x00007fcc84091e49 in  () at /usr/lib/libQt5WebKit.so.5
#5  0x00007fcc818f2b74 in std::execute_native_thread_routine(void*) (__p=0x560210d79620) at /build/gcc/src/gcc/libstdc++-v3/src/c++11/thread.cc:80
#6  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#7  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 11 (Thread 0x7fcc73d20700 (LWP 16075)):
#0  0x00007fcc815ce05f in poll () at /usr/lib/libc.so.6
#1  0x00007fcc854bd63b in  () at /usr/lib/libxcb.so.1
#2  0x00007fcc854bf37b in xcb_wait_for_event () at /usr/lib/libxcb.so.1
#3  0x00007fcc73f258f1 in  () at /usr/lib/libQt5XcbQpa.so.5
#4  0x00007fcc856cedbf in  () at /usr/lib/libQt5Core.so.5
#5  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#6  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 10 (Thread 0x7fcc747f8700 (LWP 16074)):
#0  0x00007fcc815ce05f in poll () at /usr/lib/libc.so.6
#1  0x000056020e936678 in juce::InternalRunLoop::sleepUntilNextEvent(int) (timeoutMs=2000, this=0x7fcc6c000bd0) at /usr/include/c++/10.1.0/bits/stl_iterator.h:953
#2  juce::MessageManager::dispatchNextMessageOnSystemQueue(bool) (returnIfNoPendingMessages=<optimized out>) at ../../JuceLibraryCode/modules/juce_events/native/juce_linux_Messaging.cpp:266
#3  juce::MessageManager::runDispatchLoop() (this=<optimized out>) at ../../JuceLibraryCode/modules/juce_events/messages/juce_MessageManager.cpp:128
#4  0x000056020e796ac0 in (anonymous namespace)::JuceThread::run() (this=0x560210cf03b0) at ../../../audio/Juce_plugins.cpp:1258
#5  0x000056020e8edd6f in juce::Thread::threadEntryPoint() (this=0x560210cf03b0) at ../../JuceLibraryCode/modules/juce_core/threads/juce_Thread.cpp:96
#6  0x000056020e8edfa9 in juce::juce_threadEntryPoint(void*) (userData=<optimized out>) at ../../JuceLibraryCode/modules/juce_core/threads/juce_Thread.cpp:118
#7  juce::threadEntryProc(void*) (userData=<optimized out>) at ../../JuceLibraryCode/modules/juce_core/native/juce_posix_SharedCode.h:834
#8  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#9  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 9 (Thread 0x7fcc74ff9700 (LWP 16073)):
#0  0x00007fcc816afe32 in pthread_cond_wait@@GLIBC_2.3.2 () at /usr/lib/libpthread.so.0
#1  0x000056020eb874b7 in GC_wait_marker () at pthread_support.c:2144
#2  0x000056020eb7fa8a in GC_help_marker (my_mark_no=my_mark_no@entry=6) at mark.c:1205
#3  0x000056020eb8746c in GC_mark_thread (id=<optimized out>) at pthread_support.c:377
#4  GC_mark_thread (id=<optimized out>) at pthread_support.c:339
#5  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#6  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 8 (Thread 0x7fcc757fa700 (LWP 16072)):
#0  0x00007fcc816afe32 in pthread_cond_wait@@GLIBC_2.3.2 () at /usr/lib/libpthread.so.0
#1  0x000056020eb874b7 in GC_wait_marker () at pthread_support.c:2144
#2  0x000056020eb7fa8a in GC_help_marker (my_mark_no=my_mark_no@entry=6) at mark.c:1205
#3  0x000056020eb8746c in GC_mark_thread (id=<optimized out>) at pthread_support.c:377
#4  GC_mark_thread (id=<optimized out>) at pthread_support.c:339
#5  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#6  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 7 (Thread 0x7fcc75ffb700 (LWP 16071)):
#0  0x00007fcc816afe32 in pthread_cond_wait@@GLIBC_2.3.2 () at /usr/lib/libpthread.so.0
#1  0x000056020eb874b7 in GC_wait_marker () at pthread_support.c:2144
#2  0x000056020eb7fa8a in GC_help_marker (my_mark_no=my_mark_no@entry=6) at mark.c:1205
#3  0x000056020eb8746c in GC_mark_thread (id=<optimized out>) at pthread_support.c:377
#4  GC_mark_thread (id=<optimized out>) at pthread_support.c:339
#5  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#6  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 6 (Thread 0x7fcc767fc700 (LWP 16070)):
#0  0x00007fcc816afe32 in pthread_cond_wait@@GLIBC_2.3.2 () at /usr/lib/libpthread.so.0
#1  0x000056020eb874b7 in GC_wait_marker () at pthread_support.c:2144
#2  0x000056020eb7fa8a in GC_help_marker (my_mark_no=my_mark_no@entry=6) at mark.c:1205
#3  0x000056020eb8746c in GC_mark_thread (id=<optimized out>) at pthread_support.c:377
#4  GC_mark_thread (id=<optimized out>) at pthread_support.c:339
#5  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#6  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 5 (Thread 0x7fcc76ffd700 (LWP 16069)):
#0  0x00007fcc816afe32 in pthread_cond_wait@@GLIBC_2.3.2 () at /usr/lib/libpthread.so.0
#1  0x000056020eb874b7 in GC_wait_marker () at pthread_support.c:2144
#2  0x000056020eb7fa8a in GC_help_marker (my_mark_no=my_mark_no@entry=6) at mark.c:1205
#3  0x000056020eb8746c in GC_mark_thread (id=<optimized out>) at pthread_support.c:377
#4  GC_mark_thread (id=<optimized out>) at pthread_support.c:339
#5  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#6  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 4 (Thread 0x7fcc777fe700 (LWP 16068)):
#0  0x00007fcc816afe32 in pthread_cond_wait@@GLIBC_2.3.2 () at /usr/lib/libpthread.so.0
#1  0x000056020eb874b7 in GC_wait_marker () at pthread_support.c:2144
#2  0x000056020eb7fa8a in GC_help_marker (my_mark_no=my_mark_no@entry=6) at mark.c:1205
#3  0x000056020eb8746c in GC_mark_thread (id=<optimized out>) at pthread_support.c:377
#4  GC_mark_thread (id=<optimized out>) at pthread_support.c:339
#5  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#6  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 3 (Thread 0x7fcc77fff700 (LWP 16067)):
#0  0x00007fcc816afe32 in pthread_cond_wait@@GLIBC_2.3.2 () at /usr/lib/libpthread.so.0
#1  0x000056020eb874b7 in GC_wait_marker () at pthread_support.c:2144
#2  0x000056020eb7fa8a in GC_help_marker (my_mark_no=my_mark_no@entry=6) at mark.c:1205
#3  0x000056020eb8746c in GC_mark_thread (id=<optimized out>) at pthread_support.c:377
#4  GC_mark_thread (id=<optimized out>) at pthread_support.c:339
#5  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#6  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 2 (Thread 0x7fcc7cb5c700 (LWP 16066)):
#0  0x00007fcc816b28f4 in do_futex_wait.constprop () at /usr/lib/libpthread.so.0
#1  0x00007fcc816b29f8 in __new_sem_wait_slow.constprop.0 () at /usr/lib/libpthread.so.0
#2  0x000056020e0e0a76 in cpp11onmulticore::Semaphore::wait() (this=<optimized out>) at audio/../common/sema.h:156
#3  radium::Semaphore::wait() (this=0x5602101cb2f8 <radium::g_disk_peaks_queue+56>) at audio/../common/Semaphores.hpp:79
#4  radium::BaseQueueStack<radium::DiskPeaks*>::wait() (this=0x5602101cb2c0 <radium::g_disk_peaks_queue>) at audio/../common/QueueStack.hpp:83
#5  radium::BaseQueueStack<radium::DiskPeaks*>::get() (this=0x5602101cb2c0 <radium::g_disk_peaks_queue>) at audio/../common/QueueStack.hpp:105
#6  (anonymous namespace)::DiskPeaksThread::run() (this=0x5602101db430 <(anonymous namespace)::g_disk_peaks_thread>) at audio/Peaks.cpp:47
#7  0x00007fcc856cedbf in  () at /usr/lib/libQt5Core.so.5
#8  0x00007fcc816a9422 in start_thread () at /usr/lib/libpthread.so.0
#9  0x00007fcc815d8bf3 in clone () at /usr/lib/libc.so.6

Thread 1 (Thread 0x7fcc7cffc180 (LWP 16065)):
#0  0x00007fcc816b36bf in __lll_clocklock_wait () at /usr/lib/libpthread.so.0
#1  0x00007fcc816abb59 in __pthread_mutex_lock_full () at /usr/lib/libpthread.so.0
#2  0x000056020e0e3512 in lock_player () at audio/Mixer.cpp:281
#3  lock_player_from_nonrt_thread () at audio/Mixer.cpp:372
#4  PLAYER_lock() () at audio/Mixer.cpp:488
#5  0x000056020e798aa6 in radium::(anonymous namespace)::PlayerRecursiveLock::PlayerRecursiveLock (this=<synthetic pointer>) at ../../../audio/../common/OS_Player_proc.h:177
#6  get_effect_value(SoundPlugin*, int, ValueFormat) (plugin=0x56021da43d10, effect_num=<optimized out>, value_format=<optimized out>) at ../../../audio/Juce_plugins.cpp:1648
#7  0x000056020e0dd7dc in PLUGIN_create(Patch*, SoundPluginType*, hash_t*, bool) (patch=patch@entry=0x560219055960, plugin_type=plugin_type@entry=0x56021d0f7810, plugin_state=plugin_state@entry=0x0, is_loading=is_loading@entry=false) at audio/SoundPlugin.cpp:559
#8  0x000056020e05fd5d in AUDIO_InitPatch2(Patch*, char const*, char const*, hash_t*, bool, bool, float, float) (patch=0x560219055960, type_name=0x560213e78b50 "VST", plugin_name=0x560213e78b40 "renoise_redux", audio_state=0x0, is_loading_song=<optimized out>, set_as_current=<optimized out>, x=-480, y=-128) at audio/audio_instrument.cpp:516
#9  0x000056020de606bb in PATCH_make_active_audio (y=-128, x=-480, set_as_current=true, state=<optimized out>, plugin_name=0x560213e78b40 "renoise_redux", type_name=0x560213e78b50 "VST", patch=0x560219055960) at common/patch.cpp:618
#10 PATCH_make_active_audio(Patch*, char const*, char const*, hash_t*, bool, float, float) (patch=0x560219055960, type_name=0x560213e78b50 "VST", plugin_name=0x560213e78b40 "renoise_redux", state=<optimized out>, set_as_current=<optimized out>, x=-480, y=-128) at common/patch.cpp:601
#11 0x000056020de607f5 in create_audio_patch(char const*, char const*, char const*, hash_t*, float, float, bool, bool) (type_name=type_name@entry=0x560213e78b50 "VST", plugin_name=plugin_name@entry=0x560213e78b40 "renoise_redux", name=name@entry=0x0, state=state@entry=0x0, x=-480, y=-128, is_main_pipe=is_main_pipe@entry=false, set_as_current=set_as_current@entry=true) at common/patch.cpp:640
#12 0x000056020de6092c in PATCH_create_audio(char const*, char const*, char const*, hash_t*, bool, float, float) (type_name=type_name@entry=0x560213e78b50 "VST", plugin_name=plugin_name@entry=0x560213e78b40 "renoise_redux", name=name@entry=0x0, state=state@entry=0x0, set_as_current=set_as_current@entry=true, x=<optimized out>, y=<optimized out>) at common/patch.cpp:662
#13 0x000056020df0f100 in createAudioInstrument(char const*, char const*, char const*, float, float, bool) (type_name=0x560213e78b50 "VST", plugin_name=0x560213e78b40 "renoise_redux", name=name@entry=0x0, x=-480, y=-128, set_as_current=set_as_current@entry=true) at api/api_instruments.cpp:431
#14 0x000056020df0f30c in createAudioInstrumentFromDescription(char const*, char const*, float, float, bool) (instrument_description=instrument_description@entry=0x56021da7f670 "1cmVub2lzZV9yZWR1eA==:VlNU:cmVub2lzZV9yZWR1eA==", name=0x0, name@entry=0x5602115b90f8 "", x=<optimized out>, y=<optimized out>, set_as_current=<optimized out>) at api/api_instruments.cpp:504
#15 0x000056020ddc12b2 in radium_s7_createAudioInstrumentFromDescription (radiums7_sc=0x560210fca960, radiums7_args=<optimized out>) at api/radium_s7_wrap.c:29964
#16 0x000056020dc8936a in op_safe_or_unsafe_c_fp_1 (sc=sc@entry=0x560210fca960) at bin/packages/s7/s7.c:88219
#17 0x000056020dc6ede0 in eval (sc=0x560210fca960, first_op=<optimized out>) at bin/packages/s7/s7.c:90115
#18 0x000056020dcd698d in s7_call (sc=sc@entry=0x560210fca960, func=0x7fcc726d3640, args=args@entry=0x7fcc2363c9b0) at bin/packages/s7/s7.c:52164
#19 0x000056020dc118f7 in catch_call(s7_scheme*, s7_pointer) (sc=0x560210fca960, args=0x7fcc2363c9b0) at embedded_scheme/scheme.cpp:708
#20 0x000056020dc12957 in s7extra_callFunc_void_int(func_t const*, int64_t) (func=0x7fcc238305d0, arg1=1899) at embedded_scheme/scheme.cpp:1183
#21 0x000056020dba453a in (anonymous namespace)::Callbacker::run_callbacks (this=0x56021da48610) at Qt/Qt_PopupMenu.cpp:199
#22 (anonymous namespace)::Callbacker::call_callback_and_delete (callbacker=std::shared_ptr<(anonymous namespace)::Callbacker> (use count 491919952, weak count 22017) = {...}, this=0x56021da48610) at Qt/Qt_PopupMenu.cpp:244
#23 operator() (__closure=0x7ffc43db07f0) at Qt/Qt_PopupMenu.cpp:263
#24 QtPrivate::FunctorCall<QtPrivate::IndexesList<>, QtPrivate::List<>, void, (anonymous namespace)::Callbacker::schedule_call_callback_and_delete(int, std::shared_ptr<(anonymous namespace)::Callbacker>)::<lambda()> >::call (arg=<optimized out>, f=...) at /usr/include/qt/QtCore/qobjectdefs_impl.h:146
#25 QtPrivate::Functor<(anonymous namespace)::Callbacker::schedule_call_callback_and_delete(int, std::shared_ptr<(anonymous namespace)::Callbacker>)::<lambda()>, 0>::call<QtPrivate::List<>, void> (arg=<optimized out>, f=...) at /usr/include/qt/QtCore/qobjectdefs_impl.h:256
#26 QtPrivate::QFunctorSlotObject<(anonymous namespace)::Callbacker::schedule_call_callback_and_delete(int, std::shared_ptr<(anonymous namespace)::Callbacker>)::<lambda()>, 0, QtPrivate::List<>, void>::impl(int, QtPrivate::QSlotObjectBase *, QObject *, void **, bool *) (which=<optimized out>, this_=0x7ffc43db07e0, r=<optimized out>, a=<optimized out>, ret=<optimized out>) at /usr/include/qt/QtCore/qobjectdefs_impl.h:443
#27 0x00007fcc858e78d3 in  () at /usr/lib/libQt5Core.so.5
#28 0x00007fcc858da06f in QObject::event(QEvent*) () at /usr/lib/libQt5Core.so.5
#29 0x00007fcc84d4d652 in QApplicationPrivate::notify_helper(QObject*, QEvent*) () at /usr/lib/libQt5Widgets.so.5
#30 0x00007fcc858ae2c2 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () at /usr/lib/libQt5Core.so.5
#31 0x00007fcc85904475 in QTimerInfoList::activateTimers() () at /usr/lib/libQt5Core.so.5
#32 0x00007fcc85904d22 in  () at /usr/lib/libQt5Core.so.5
#33 0x00007fcc8552d43c in g_main_context_dispatch () at /usr/lib/libglib-2.0.so.0
#34 0x00007fcc8557afa9 in  () at /usr/lib/libglib-2.0.so.0
#35 0x00007fcc8552c221 in g_main_context_iteration () at /usr/lib/libglib-2.0.so.0
#36 0x00007fcc859050f1 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () at /usr/lib/libQt5Core.so.5
#37 0x00007fcc858acbfc in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () at /usr/lib/libQt5Core.so.5
#38 0x00007fcc858b5096 in QCoreApplication::exec() () at /usr/lib/libQt5Core.so.5
#39 0x000056020db8fa18 in radium_main(char const*) (arg=arg@entry=0x7fcc719598f4 "") at Qt/Qt_Main.cpp:3248
#40 0x000056020def605c in init_radium(char const*, PyObject*) (arg=0x7fcc719598f4 "", gkf=0x7fcc702076d0) at api/api_common.cpp:79
#41 0x000056020dee982c in _wrap_init_radium (self=<optimized out>, args=0x7fcc70656780) at api/radium_wrap.c:585
#42 0x00007fcc865670bb in PyEval_EvalFrameEx () at /usr/lib/libpython2.7.so.1.0
#43 0x00007fcc86565cf6 in PyEval_EvalCodeEx () at /usr/lib/libpython2.7.so.1.0
#44 0x00007fcc86566ba7 in PyEval_EvalCode () at /usr/lib/libpython2.7.so.1.0
#45 0x00007fcc865dafd2 in run_mod () at /usr/lib/libpython2.7.so.1.0
#46 0x00007fcc865db07a in PyRun_FileExFlags () at /usr/lib/libpython2.7.so.1.0
#47 0x00007fcc865e2f67 in builtin_execfile.lto_priv () at /usr/lib/libpython2.7.so.1.0
#48 0x00007fcc865670bb in PyEval_EvalFrameEx () at /usr/lib/libpython2.7.so.1.0
#49 0x00007fcc86565cf6 in PyEval_EvalCodeEx () at /usr/lib/libpython2.7.so.1.0
#50 0x00007fcc86566ba7 in PyEval_EvalCode () at /usr/lib/libpython2.7.so.1.0
#51 0x00007fcc865dafd2 in run_mod () at /usr/lib/libpython2.7.so.1.0
#52 0x00007fcc8654fa05 in PyRun_StringFlags () at /usr/lib/libpython2.7.so.1.0
#53 0x00007fcc865db8ef in PyRun_SimpleStringFlags () at /usr/lib/libpython2.7.so.1.0
#54 0x000056020da94135 in main(int, char**) (argc=<optimized out>, argv=<optimized out>) at Qt/Qt_Main.cpp:4129

Posted: **29 May 2020 16:27**

Thank you. This is really strange though. Seems like all your jack threads, except one, are gone. The audio thread is not there. Which version of jack are you running?

The freeze seems to be caused by a deadlock where the main thread waits for the audio thread to finish, but the audio thread is gone.

I don't have experience with gdb attaching to an existing process, so maybe this can happen. Doing some research now.

Radium

Getting Less Crashes in Radium

Re: Getting Less Crashes in Radium

Re: Getting Less Crashes in Radium

Re: Getting Less Crashes in Radium

Re: Getting Less Crashes in Radium

Re: Getting Less Crashes in Radium

Re: Getting Less Crashes in Radium

Re: Getting Less Crashes in Radium

Re: Getting Less Crashes in Radium

Re: Getting Less Crashes in Radium

Re: Getting Less Crashes in Radium