[3dem] visualization of data from a remote server

Sun Feb 3 11:38:54 PST 2019

Maybe a quick summary of some of the issues for CryoEM:

1) remote X sessions
pros:
 - simple for any machine with an X-client
cons:
 - very bandwidth intensive, poor interactivity on slow links
 - often requires tricky configuration work to make OpenGL function at all
 - openGL badly broken on many clients, and slightly broken on most, not always immediately obvious

2) compressed remote X sessions (NX, X2GO, etc.)
pros:
 - compression gives much improved performance
 - may support OpenGL
cons:
 - historically this has gone through many cycles of working/not working with OpenGL
 - compression based on X protocol. Great for things like word-processors, but not great for graphics/image intensive work

3) TurboVNC/VirtualGL
pros:
 - developed by NSF supercomputing centers specifically for remote 3-D display
 - sends compressed image data tiles, so all rendering done on the server
 - works perfectly almost 100% of the time once configured
 - cross-platform client
 - server set up on many clusters
 - can be tunneled through SSH easily
cons:
 - rendering done server-side, so more CPU/GPU load on the server (in practice I've never seen a problem)
 - needs to be setup on server and client. Root privileges required on server
 - X2GO may be more efficient for programs which aren't graphics intensive

4) Screen-sharing solutions (TeamViewer and others)
pros:
 - easy, just works
cons:
 - usually costs $
 - data routed through 3rd party service
 - network administrators do NOT like this, as it bypasses established network security and can expose the entire institutional network
 - like TurboVNC, involves compression, but may not be under user control (bandwidth sensitive)

--------------------------------------------------------------------------------------
Steven Ludtke, Ph.D. <sludtke at bcm.edu<mailto:sludtke at bcm.edu>>                      Baylor College of Medicine
Charles C. Bell Jr., Professor of Structural Biology
Dept. of Biochemistry and Molecular Biology                      (www.bcm.edu/biochem<http://www.bcm.edu/biochem>)
Academic Director, CryoEM Core                                        (cryoem.bcm.edu<http://cryoem.bcm.edu>)
Co-Director CIBR Center                                    (www.bcm.edu/research/cibr<http://www.bcm.edu/research/cibr>)

On Feb 3, 2019, at 1:08 PM, frankg at bgu.ac.il<mailto:frankg at bgu.ac.il> wrote:

***CAUTION:*** This email is not from a BCM Source. Only click links or open attachments you know are safe.
________________________________
Hi,

We recently installed X2Go on our servers using the lightweight XFCE graphical environment.
This works very well for us.
There are versions of X2Go clients for Linux, W10 and Mac.

Gabriel A. Frank,
BGU

From: 3dem [mailto:3dem-bounces at ncmir.ucsd.edu] On Behalf Of Amar Dhananjai Parvate
Sent: יום א 03 פברואר 2019 20:35
To: Michael Elbaum <michael.elbaum at weizmann.ac.il<mailto:michael.elbaum at weizmann.ac.il>>; 3dem at ncmir.ucsd.edu<mailto:3dem at ncmir.ucsd.edu>
Subject: Re: [3dem] visualization of data from a remote server

Dear All
I am very glad to know i am not the only one with remote log in GUI visualization problems and ... even better that ppl on this forum have some solutions.
Here is what i am trying to do and of course its not working...
1) My workstation is a linux machine with Centos 7. I can access both single particle and tomography software through my home directory.
2) I have a Windows 10 laptop where i have installed my institute recommended Putty and Xming client for remote login
3) While off campus, my VPN works fine, i can log in to my workstation, access home directory and run anything command line properly.
4) Issue comes when anything GUI needs to be displayed. I have followed several tutorials for Putty and Xming, tried loging in to the computer via IP or user at home.edu<mailto:user at home.edu>. Have tried several local hosts for X11 forwarding... and it doesnt seem to work.
I have even tried to edit the ssh-config file. where the X11 forwarding is turned off by default.
Basically if i try to open a .mrc file via say 3dmod, i see the file being read in the main window but the connection is refused while displaying the zap window.

Be glad if anyone can share any solutions on this regard...

P.S. - The Putty+X forwarding worked for my colleague who basically has the same laptop as mine, and we are in the same lab, trying to access the same set of imod GUIs.  🤔

Amar Parvate
Purdue University
Department of Biological Sciences

________________________________
From: 3dem <3dem-bounces at ncmir.ucsd.edu<mailto:3dem-bounces at ncmir.ucsd.edu>> on behalf of Michael Elbaum <michael.elbaum at weizmann.ac.il<mailto:michael.elbaum at weizmann.ac.il>>
Sent: Sunday, February 3, 2019 10:27 AM
To: 3dem at ncmir.ucsd.edu<mailto:3dem at ncmir.ucsd.edu>
Subject: [3dem] visualization of data from a remote server

Hi all,

I'm normally using a powerful server for image processing, and then I look at the data by remote ssh with X forwarding. Recently I upgraded the linux OS on my laptop (to Ubuntu 18.04) and all sorts of programs stopped working this way, including 3dmod. It turned out to be a much broader problem having to do with OpenGL. Indirect rendering, which is what we need, is turned off by default in newer versions of X. I assume others have encountered such trouble as well. Since it took quite some digging to find a solution I'm posting it here.

For those who boot into a command line interface with a black screen and start an X session manually, (e.g., startx -- :2) the fix is simple.
Edit the file /usr/bin/startx and change the line defaultclientargs="" to defaultclientargs="+iglx"
+iglx allows indirect rendering by OpenGL applications.
This is not relevant to most of us.

For those who boot the computer and expect to see the graphical display right away it's a bit trickier. Ubuntu uses X or Wayland for the graphical interface, depending on the version. The solution I found is only for X, so we have to revert to that first. It's done in two steps, as root:
1) In the file /etc/gdm3/custom.conf, uncomment (erase the hash) the line #WaylandEnable=false. (In Ubuntu it should be enough to start GNOME in Xorg, which is one of the login options.)
2) Create a new file /usr/share/X11/xorg.conf.d/50-iglx.conf:
Section "ServerFlags"
Option "AllowIndirectGLX" "on"
    Option "IndirectGLX" "on"
EndSection
Reboot and it should work. at your own risk, of course...
Other distributions might put these files in different places.

Here are some useful resources:
https://www.visitusers.org/index.php?title=Re-enabling_INdirect_glx_on_your_X_server<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.visitusers.org_index.php-3Ftitle-3DRe-2Denabling-5FINdirect-5Fglx-5Fon-5Fyour-5FX-5Fserver&d=DwMGaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=qY9gTMBGM37MS6-wNHhwW5WhlSPN9gt1MjWTiO0Y22Y&s=LK4vpKWDMPgF6hj-Tw_TcRRUYUK8yQNvD7g3hS-Wa3o&e=>
https://lists.freedesktop.org/archives/xorg/2016-May/058072.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.freedesktop.org_archives_xorg_2016-2DMay_058072.html&d=DwMGaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=qY9gTMBGM37MS6-wNHhwW5WhlSPN9gt1MjWTiO0Y22Y&s=qtB13C4ftJryMYlU48WVrGF-Q8ikzbV_TJCEbD-9zWU&e=>
https://askubuntu.com/questions/745135/how-to-enable-indirect-glx-contexts-iglx-in-ubuntu-14-04-lts-with-nvidia-gfx/747088<https://urldefense.proofpoint.com/v2/url?u=https-3A__askubuntu.com_questions_745135_how-2Dto-2Denable-2Dindirect-2Dglx-2Dcontexts-2Diglx-2Din-2Dubuntu-2D14-2D04-2Dlts-2Dwith-2Dnvidia-2Dgfx_747088&d=DwMGaQ&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=qY9gTMBGM37MS6-wNHhwW5WhlSPN9gt1MjWTiO0Y22Y&s=dmm1y_NIZbPi1OHVN5WGTrkH8LDK0IFknU-3FYc3Tak&e=>

There seems to be a similar issue with XQuartz on the Mac, which is addressed by reverting to the last version that worked.

There was another instruction to add in the file /usr/share/lightdm/lightdm.conf.d/50-xserver-command.conf
[SeatDefaults]
# Dump core
xserver-command=X -core +iglx
but this didn't work for me.

As for Wayland, typing Xwayland -h shows that the options include +iglx, but I wasn't able to find a config file for passing it startup options.

A longer term solution seems to be VirtualGL or TurboVNC. Does anyone have those working with 3dem visualization software?

regards,
Michael
_______________________________________________
3dem mailing list
3dem at ncmir.ucsd.edu<mailto:3dem at ncmir.ucsd.edu>
https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.ncmir.ucsd.edu_mailman_listinfo_3dem&d=DwICAg&c=ZQs-KZ8oxEw0p81sqgiaRA&r=Dk5VoQQ-wINYVssLMZihyC5Dj_sWYKxCyKz9E4Lp3gc&m=qY9gTMBGM37MS6-wNHhwW5WhlSPN9gt1MjWTiO0Y22Y&s=OOiMvjHx9Um0uPBBumyd_FM2TbKgtQbFuoxED5ii8to&e=

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ncmir.ucsd.edu/pipermail/3dem/attachments/20190203/90c3a7e3/attachment-0001.html>