Jupyter Notebooks
Vincent Jeanselme
vjeansel at andrew.cmu.edu
Tue Nov 6 22:06:53 EST 2018
Thank you ! Uncommenting the line: c.NotebookNotary.db_file worked for
my Jupyter
Le 06/11/2018 à 21:39, Chirag Nagpal a écrit :
> Ok I think I have a quick fix for people struggling with jupyter
> notebook on NFS.
>
> There are essentially two parts to the problem. The first one deals
> with forcing jupyter to create its SQLite db in your scratch
> directory. The second part is the ipython directory.
>
>
> Part 1:
>
> Step 1 : First ssh into the Auton Computing Environment and run
>
> $jupyter notebook --generate-config
>
> This would create a config file 'jupyter_notebook_config.py' in
> ~/.jupyter . ( Note that ~ this is /zfsauton/home/<username> )
>
> Step 2: Edit the file above with your favourite text editor and add
> (or uncomment) the following line
>
> c.NotebookNotary.db_file='/home/scratch/<username>/jupyter.log'
>
> replace <username> with your auton username.
>
> If you executed part 1 correctly, you should be able to create a new
> Notebook, and stop the Ipython server using control-c. However, you
> will not be able to connect Jupyter to Ipython for which you need to
> perform part 2
>
> Part 2:
>
> $export IPYTHONDIR=/home/scratch/<username>/ipython
>
> as always replace<username> with your auton username.
>
> you can also add this to .bashrc for next time ;)
>
> Thats it!
>
> It should work now!
>
> Chirag
>
>
>
>
>
>
>
>
> On Tue, Nov 6, 2018 at 8:43 PM, Chirag Nagpal <chiragn at cs.cmu.edu
> <mailto:chiragn at cs.cmu.edu>> wrote:
>
> I am working on fixing this.
>
> It is indeed SQLite and NFS not talking to each other which is the
> problem. I am able to get jupyter to behave somewhat better by
> forcing the SQLite server to be in the scratch instead if the NFS.
> This requires changing some default flags for jupyter.
>
> Its not completely fixed yet, but I will get back with an update
> soon.
>
> Chirag
>
>
>
> On Tue, Nov 6, 2018 at 8:39 PM, Predrag Punosevac
> <predragp at andrew.cmu.edu <mailto:predragp at andrew.cmu.edu>> wrote:
>
> Chirag Nagpal <chiragn at cs.cmu.edu <mailto:chiragn at cs.cmu.edu>>
> wrote:
>
> > Predrag: I am able to reproduce the error on the scratch
> directory too.
> >
>
> I am sure you guys have a problem but one doesn't need Jupyter
> to do
> actual programming in Python. I am saying this because there are
> handful of you who are affected by this behavior (God knows
> what could
> have caused possibly even regressions by newer version of
> packages) and
> I am the only firefighter currently without bandwidth to deal
> with such
> wild fires.
>
> > Vincent, Matt: are you using the anaconda jupyter? it could
> be an anaconda
> > upgrade thats responsible ?
> >
> > Chirag
> >
> > On Tue, Nov 6, 2018 at 4:45 PM, Vincent Jeanselme
> <vjeansel at andrew.cmu.edu <mailto:vjeansel at andrew.cmu.edu>>
> > wrote:
> >
> > > Even after changing my bashrc with export
> IPYTHONDIR=/home/scratch/$USER/.ipython
> > > and reinstalling my jupyter, it still does not seem to
> work. I also have an
> > > issue with git, I am no longer able to pull from the server.
> > >
>
>
> The git issue is environmental variable issue which is caused
> by the
> fact that there is only one user on Gogs server (git) and all
> accounts
> are just aliases with their own ssh-keys to this account. I
> don't use
> and know enough about git but those nasty files in your
>
> reponame/.git
>
> folder
>
> which look like
>
> predragp at lov3$ ls
> branches description HEAD index logs ORIG_HEAD refs
> config FETCH_HEAD hooks info objects packed-refs
>
> apparently get populated in different ways depending on the
> login. So
> for example if I ssh to one of the computing nodes from home I
> get this
>
>
> predragp at lov3$ git pull
> Host key fingerprint is
> SHA256:XEkrneUFkkAPyf0gMOQKa3amiAp3QrXmu1x+sIJMcsA
> +---[ECDSA 256]---+
> | .++++o.o.. |
> | . .+=+B . |
> |. . . . ..Bo. |
> |.E + o . o . |
> | .= * S |
> |o=o* . |
> |+=+.o .o |
> |o.o+.+. . |
> |. +o.. |
> +----[SHA256]-----+
> remote: Enumerating objects: 34, done.
> remote: Counting objects: 100% (34/34), done.
> remote: Compressing objects: 100% (21/21), done.
> remote: Total 22 (delta 8), reused 0 (delta 0)
> Unpacking objects: 100% (22/22), done.
> From ssh://git:/predragp/ansible
> cfc10cf..71a9aec master -> origin/master
> Updating cfc10cf..71a9aec
> Fast-forward
> Linux/autofs/etc/auto.nfs | 18
> +++++++++++++++++
> Linux/ldap/etc/openldap/{certs => cacerts}/ca.crt | 0
> Linux/ldap/etc/openldap/ldap.conf | 6 +++---
> Linux/ldap/etc/sssd/sssd.conf | 2 +-
> Linux/ldap/ldap.yaml | 24
> +++++++++++++++--------
> 5 files changed, 38 insertions(+), 12 deletions(-)
> rename Linux/ldap/etc/openldap/{certs => cacerts}/ca.crt (100%)
>
>
> If I ssh to my desktop I get
>
> predragp at lake$ git pull
> Host key fingerprint is
> SHA256:XEkrneUFkkAPyf0gMOQKa3amiAp3QrXmu1x+sIJMcsA
> +---[ECDSA 256]---+
> | .++++o.o.. |
> | . .+=+B . |
> |. . . . ..Bo. |
> |.E + o . o . |
> | .= * S |
> |o=o* . |
> |+=+.o .o |
> |o.o+.+. . |
> |. +o.. |
> +----[SHA256]-----+
> Password for git at git.int.autonlab.org
> <mailto:git at git.int.autonlab.org>:
>
> which is the indication that my .ssh/config file and the
> ssh-key were
> not read even though
>
> Host git
> HostName git.int.autonlab.org <http://git.int.autonlab.org>
> Port 2222
> User git
> IdentityFile /home/predragp/.ssh/git_rsa
>
> However if I log from the terminal to my desktop I don't have
> a Git
> issue.
>
>
> Best,
> Predrag
>
>
>
>
>
>
> > >
> > > On 11/6/18 3:14 PM, Predrag Punosevac wrote:
> > >
> > >> Matthew Barnes <mbarnes1 at andrew.cmu.edu
> <mailto:mbarnes1 at andrew.cmu.edu>> wrote:
> > >>
> > >> Also having this problem. Trying to create a new notebook
> hangs on
> > >>> "Creating new notebook in", and unable to open old
> notebooks. Anyone's
> > >>> setup currently working?
> > >>>
> > >>> Jupyter Notebook is using sqlite database to store the
> info. Unless you
> > >> explicitly force Jupyter to create the database on the
> scratch directory
> > >> the database is stored on the NFS share. There is nothing
> worse one can
> > >> do in terms of data consistency than put a database or a
> private Git
> > >> repo (talking about the server) onto the NFS. The
> datebase was left in
> > >> inconsistent state after the file server was rebooted.
> You have to clear
> > >> it and possibly recreate the database to be able to use
> Jupyter
> > >> Notebook.
> > >>
> > >> Best,
> > >> Predrag
> > >>
> > >>
> > >> On Tue, Nov 6, 2018 at 12:44 PM Vincent Jeanselme <
> > >>> vjeansel at andrew.cmu.edu <mailto:vjeansel at andrew.cmu.edu>>
> > >>> wrote:
> > >>>
> > >>> Hello,
> > >>>>
> > >>>> Has anyone faced problems with running Jupyter Notebook
> since yesterday
> > >>>> ?
> > >>>> Did you remember what was the change to operate after
> the last reboot of
> > >>>> the sqlite database ?
> > >>>>
> > >>>> Thank you,
> > >>>>
> > >>>> Vincent
> > >>>>
> > >>>> --
> > >>>> Vincent Jeanselme
> > >>>> -----------------
> > >>>> Analyst Researcher
> > >>>> Auton Lab - Robotics Institute
> > >>>> Carnegie Mellon University
> > >>>>
> > >>>>
> > >>>> --
> > > Vincent Jeanselme
> > > -----------------
> > > Analyst Researcher
> > > Auton Lab - Robotics Institute
> > > Carnegie Mellon University
> > >
> > >
> >
> >
> > --
> >
> > *Chirag Nagpal* Graduate Student, Language Technologies
> Institute
> > School of Computer Science
> > Carnegie Mellon University
> > cs.cmu.edu/~chiragn <http://cs.cmu.edu/~chiragn>
>
>
>
>
> --
> *Chirag Nagpal
> *Graduate Student, Language Technologies Institute
> School of Computer Science
> Carnegie Mellon University
> cs.cmu.edu/~chiragn <http://cs.cmu.edu/~chiragn>
>
>
>
>
> --
> *Chirag Nagpal
> * Graduate Student, Language Technologies Institute
> School of Computer Science
> Carnegie Mellon University
> cs.cmu.edu/~chiragn <http://cs.cmu.edu/~chiragn>
--
Vincent Jeanselme
-----------------
Analyst Researcher
Auton Lab - Robotics Institute
Carnegie Mellon University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20181106/5034d12f/attachment-0001.html>
More information about the Autonlab-users
mailing list