• BBQ Sprint

  • sprinters!

    from whit on Mar 09, 2007 04:53 AM
    hey guys,
    count down to the start here!
    
    just checked the subscriptions to the bbq-sprint list and it looks likes 
    only about half of the folks coming to the sprint have joined the list 
    and I've gotten little or no feedback about what sort of things people 
    are interested in working on or learning. so....
    
    please join the list:
    
    http://www.openplans.org/projects/bbq-sprint/lists/bbq-sprint
    
    and if have joined send an email introducing yourself and what you are 
    interested in taking home from this sprint. 
    
    In other new related to topics, Michel Pelletier has expressed an 
    interest in expanding some work he's done with python and Xapian(a 
    industrial strength indexing system similar to lucene) out into either a 
    zope integration or a full REST searching service.   This would be a 
    great way to learn about xapian(http://www.xapian.org/)  and how to 
    extend searching for zope and plone.
    
    -w
    
    -- 
    
    ------ d. whit morriss ------
    - senior engineer, opencore -
    - http://www.openplans.org  -
    - m: 415-710-8975           -
    
    "If you don't know where you are,   
     you don't know anything at all"                                             
    
    Dr. Edgar Spencer, Ph.D., 1995
    
    
    
    Thread Outline:
  • Re: sprinters!

    from ianb on Mar 09, 2007 07:11 AM
    whit wrote:
    > hey guys,
    > count down to the start here!
    > 
    > just checked the subscriptions to the bbq-sprint list and it looks likes 
    > only about half of the folks coming to the sprint have joined the list 
    > and I've gotten little or no feedback about what sort of things people 
    > are interested in working on or learning. so....
    > 
    > please join the list:
    > 
    > http://www.openplans.org/projects/bbq-sprint/lists/bbq-sprint
    > 
    > and if have joined send an email introducing yourself and what you are 
    > interested in taking home from this sprint.
    > In other new related to topics, Michel Pelletier has expressed an 
    > interest in expanding some work he's done with python and Xapian(a 
    > industrial strength indexing system similar to lucene) out into either a 
    > zope integration or a full REST searching service.   This would be a 
    > great way to learn about xapian(http://www.xapian.org/)  and how to 
    > extend searching for zope and plone.
    
    Last night I saw a little presentation on a little new project for 
    pylucene as well: http://code.google.com/p/grassyknoll/
    
    It's pretty rough, and I don't really know how Xapian and Lucene 
    compare.  But another search thing for the mix.  Pylucene isn't terribly 
    friendly, so I wouldn't recommend running it in the Zope process, only 
    as a REST service in a separate process.
    
    
    Oh, but more about me!  I'm interested in working on Deliverance 
    integration.  Deliverance is a theming intermediary -- it gets HTML 
    pages from somewhere (say, Zope or Plone, but could be anything) and 
    applies a "theme", i.e., the wrapper (layout, navigation, CSS, etc). 
    Right now we have some setups running Deliverance as an HTTP proxy -- 
    which is useful, but adds another stage to setup.
    
    So, I'd like to figure out how people can use Deliverance in Zope and/or 
    Plone with much less setup.  I'll also show people how Deliverance 
    works, some about why it works the way it does, as we try to figure out 
    the best strategy for deployment.  Help from someone with knowledge of 
    Zope's innards would be helpful, specifically how to do transformations 
    without entirely pulling apart Zope.
    
    Deliverance: http://openplans.org/projects/deliverance
    
       Ian
    
    • Re: sprinters!

      from chrism on Mar 09, 2007 07:13 AM
      On Mar 9, 2007, at 1:11 PM, Ian Bicking wrote:
      
      > So, I'd like to figure out how people can use Deliverance in Zope  
      > and/or Plone with much less setup.  I'll also show people how  
      > Deliverance works, some about why it works the way it does, as we  
      > try to figure out the best strategy for deployment.  Help from  
      > someone with knowledge of Zope's innards would be helpful,  
      > specifically how to do transformations without entirely pulling  
      > apart Zope.
      
      Roger.
      
      - C
      
      
    • Re: sprinters!

      from tseaver on Mar 09, 2007 09:48 AM
      -----BEGIN PGP SIGNED MESSAGE-----
      Hash: SHA1
      
      Ian Bicking wrote:
      > whit wrote:
      >> hey guys,
      >> count down to the start here!
      >>
      >> just checked the subscriptions to the bbq-sprint list and it looks likes
      >> only about half of the folks coming to the sprint have joined the list
      >> and I've gotten little or no feedback about what sort of things people
      >> are interested in working on or learning. so....
      >>
      >> please join the list:
      >>
      >> http://www.openplans.org/projects/bbq-sprint/lists/bbq-sprint
      >>
      >> and if have joined send an email introducing yourself and what you are
      >> interested in taking home from this sprint.
      >> In other new related to topics, Michel Pelletier has expressed an
      >> interest in expanding some work he's done with python and Xapian(a
      >> industrial strength indexing system similar to lucene) out into either a
      >> zope integration or a full REST searching service.   This would be a
      >> great way to learn about xapian(http://www.xapian.org/)  and how to
      >> extend searching for zope and plone.
      >
      > Last night I saw a little presentation on a little new project for
      > pylucene as well: http://code.google.com/p/grassyknoll/
      >
      > It's pretty rough, and I don't really know how Xapian and Lucene
      > compare.  But another search thing for the mix.  Pylucene isn't terribly
      > friendly, so I wouldn't recommend running it in the Zope process, only
      > as a REST service in a separate process.
      
      While we're at it, I have an intereset in plugging in SOLR (which is a
      saner, REST-like wrapper around the Lucene engine).  Indexing is done
      via simple POST requests, queries are just gets, with easy-to-parse POX
      document formats.
      
      It might be cool to define the application-side API for an external
      search engine, and then implement it via each of the three backends.
      For one thing, that would let us test each one using identical workloads.
      
      
      Tres.
      - --
      ===================================================================
      Tres Seaver          +1 540-429-0999         tseaver@...
      Agendaless Consulting                         http://agendaless.com
      -----BEGIN PGP SIGNATURE-----
      Version: GnuPG v1.4.2.2 (GNU/Linux)
      Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
      
      iD8DBQFF8cgtFXKVXuSL+CMRAsWVAJ9wKDQwQmucM03QKjM+Pgc6/34hYwCfel5G
      MVHPH4MFNRGZCaKCSI5MWoo=
      =XBcm
      -----END PGP SIGNATURE-----
      
      • Re: sprinters!

        from ianb on Mar 09, 2007 10:37 AM
        Tres Seaver wrote:
        >> It's pretty rough, and I don't really know how Xapian and Lucene
        >> compare.  But another search thing for the mix.  Pylucene isn't terribly
        >> friendly, so I wouldn't recommend running it in the Zope process, only
        >> as a REST service in a separate process.
        > 
        > While we're at it, I have an intereset in plugging in SOLR (which is a
        > saner, REST-like wrapper around the Lucene engine).  Indexing is done
        > via simple POST requests, queries are just gets, with easy-to-parse POX
        > document formats.
        
        Yeah, we were looking at Solr a little too (merely sideways glances, our 
        attention was hardly substantial).  In terms of scope it's basically the 
        same as Grassy Knoll.  In fact, Grassy Knoll probably should provide the 
        same API as Solr.  The only real advantage of Grassy Knoll is that, for 
        some people (like myself), it would be easier to deploy.  But since 
        Pylucene is a bit wonky, I'd still be inclined to deploy it in its own 
        process.
        
        I do hear that Pylucene is faster than Java Lucene, I would guess 
        because Lucene compiles well with GCJ.  But I wouldn't use that as a 
        criteria when making a choice.
        
        > It might be cool to define the application-side API for an external
        > search engine, and then implement it via each of the three backends.
        > For one thing, that would let us test each one using identical workloads.
        
        That would be nice.  I assume that would be a catalog that forwards 
        requests and updates?
        
           Ian
        
        • Re: sprinters!

          from tseaver on Mar 10, 2007 07:49 AM
          -----BEGIN PGP SIGNED MESSAGE-----
          Hash: SHA1
          
          Ian Bicking wrote:
          > Tres Seaver wrote:
          >>> It's pretty rough, and I don't really know how Xapian and Lucene
          >>> compare.  But another search thing for the mix.  Pylucene isn't terribly
          >>> friendly, so I wouldn't recommend running it in the Zope process, only
          >>> as a REST service in a separate process.
          >> While we're at it, I have an intereset in plugging in SOLR (which is a
          >> saner, REST-like wrapper around the Lucene engine).  Indexing is done
          >> via simple POST requests, queries are just gets, with easy-to-parse POX
          >> document formats.
          >
          > Yeah, we were looking at Solr a little too (merely sideways glances, our
          > attention was hardly substantial).  In terms of scope it's basically the
          > same as Grassy Knoll.  In fact, Grassy Knoll probably should provide the
          > same API as Solr.  The only real advantage of Grassy Knoll is that, for
          > some people (like myself), it would be easier to deploy.  But since
          > Pylucene is a bit wonky, I'd still be inclined to deploy it in its own
          > process.
          >
          > I do hear that Pylucene is faster than Java Lucene, I would guess
          > because Lucene compiles well with GCJ.  But I wouldn't use that as a
          > criteria when making a choice.
          >
          >> It might be cool to define the application-side API for an external
          >> search engine, and then implement it via each of the three backends.
          >> For one thing, that would let us test each one using identical workloads.
          >
          > That would be nice.  I assume that would be a catalog that forwards
          > requests and updates?
          
          I don't know, for sure, whether it would be a separate catalog, or an
          index added to an existing catalog, or just a pluggable standalone
          application ("utility", in Zope3 speak) with a well-defined API.
          
          
          Tres.
          - --
          ===================================================================
          Tres Seaver          +1 540-429-0999         tseaver@...
          Agendaless Consulting                         http://agendaless.com
          -----BEGIN PGP SIGNATURE-----
          Version: GnuPG v1.4.2.2 (GNU/Linux)
          Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
          
          iD8DBQFF8v3MFXKVXuSL+CMRApSWAJwLk3/8THajl5A/Lijp7D75OEwq2gCdGFTD
          lZpky0TpbE77GppKNwVZqgk=
          =qmg2
          -----END PGP SIGNATURE-----
          
          • Re: sprinters!

            from whit on Mar 10, 2007 08:27 AM
            Tres Seaver wrote:
            > -----BEGIN PGP SIGNED MESSAGE-----
            > Hash: SHA1
            >
            > Ian Bicking wrote:
            >   
            >> Tres Seaver wrote:
            >>     
            >>>> It's pretty rough, and I don't really know how Xapian and Lucene
            >>>> compare.  But another search thing for the mix.  Pylucene isn't terribly
            >>>> friendly, so I wouldn't recommend running it in the Zope process, only
            >>>> as a REST service in a separate process.
            >>>>         
            >>> While we're at it, I have an intereset in plugging in SOLR (which is a
            >>> saner, REST-like wrapper around the Lucene engine).  Indexing is done
            >>> via simple POST requests, queries are just gets, with easy-to-parse POX
            >>> document formats.
            >>>       
            >> Yeah, we were looking at Solr a little too (merely sideways glances, our
            >> attention was hardly substantial).  In terms of scope it's basically the
            >> same as Grassy Knoll.  In fact, Grassy Knoll probably should provide the
            >> same API as Solr.  The only real advantage of Grassy Knoll is that, for
            >> some people (like myself), it would be easier to deploy.  But since
            >> Pylucene is a bit wonky, I'd still be inclined to deploy it in its own
            >> process.
            >>
            >> I do hear that Pylucene is faster than Java Lucene, I would guess
            >> because Lucene compiles well with GCJ.  But I wouldn't use that as a
            >> criteria when making a choice.
            >>
            >>     
            >>> It might be cool to define the application-side API for an external
            >>> search engine, and then implement it via each of the three backends.
            >>> For one thing, that would let us test each one using identical workloads.
            >>>       
            >> That would be nice.  I assume that would be a catalog that forwards
            >> requests and updates?
            >>     
            >
            > I don't know, for sure, whether it would be a separate catalog, or an
            > index added to an existing catalog, or just a pluggable standalone
            > application ("utility", in Zope3 speak) with a well-defined API.
            >
            >
            >   
            tarek and michelp put together a standard interface for searching but 
            michelp can't find it. 
            
            Nuxeo has apparently created a way to talk to lucene in a transactionaly 
            safe way using hooks in ZODB3.6.
            
            a utility seems like the easiest way to jump between micro-app something 
            that runs inside zope.
            
            -w
            
            
            -- 
            
            ------ d. whit morriss ------
            - senior engineer, opencore -
            - http://www.openplans.org  -
            - m: 415-710-8975           -
            
            "If you don't know where you are,   
             you don't know anything at all"                                             
            
            Dr. Edgar Spencer, Ph.D., 1995
            
            
            
            • Re: sprinters!

              from alecpm on Mar 10, 2007 08:53 AM
              On 3/10/07, whit <d.w.morriss@...> wrote:
              > Tres Seaver wrote:
              > > -----BEGIN PGP SIGNED MESSAGE-----
              > > Hash: SHA1
              > >
              > > Ian Bicking wrote:
              > >
              > >> Tres Seaver wrote:
              > >>
              > >>>> It's pretty rough, and I don't really know how Xapian and Lucene
              > >>>> compare.  But another search thing for the mix.  Pylucene isn't terribly
              > >>>> friendly, so I wouldn't recommend running it in the Zope process, only
              > >>>> as a REST service in a separate process.
              > >>>>
              > >>> While we're at it, I have an intereset in plugging in SOLR (which is a
              > >>> saner, REST-like wrapper around the Lucene engine).  Indexing is done
              > >>> via simple POST requests, queries are just gets, with easy-to-parse POX
              > >>> document formats.
              > >>>
              > >> Yeah, we were looking at Solr a little too (merely sideways glances, our
              > >> attention was hardly substantial).  In terms of scope it's basically the
              > >> same as Grassy Knoll.  In fact, Grassy Knoll probably should provide the
              > >> same API as Solr.  The only real advantage of Grassy Knoll is that, for
              > >> some people (like myself), it would be easier to deploy.  But since
              > >> Pylucene is a bit wonky, I'd still be inclined to deploy it in its own
              > >> process.
              > >>
              > >> I do hear that Pylucene is faster than Java Lucene, I would guess
              > >> because Lucene compiles well with GCJ.  But I wouldn't use that as a
              > >> criteria when making a choice.
              > >>
              > >>
              > >>> It might be cool to define the application-side API for an external
              > >>> search engine, and then implement it via each of the three backends.
              > >>> For one thing, that would let us test each one using identical workloads.
              > >>>
              > >> That would be nice.  I assume that would be a catalog that forwards
              > >> requests and updates?
              > >>
              > >
              > > I don't know, for sure, whether it would be a separate catalog, or an
              > > index added to an existing catalog, or just a pluggable standalone
              > > application ("utility", in Zope3 speak) with a well-defined API.
              > >
              > >
              > >
              > tarek and michelp put together a standard interface for searching but
              > michelp can't find it.
              >
              > Nuxeo has apparently created a way to talk to lucene in a transactionaly
              > safe way using hooks in ZODB3.6.
              >
              > a utility seems like the easiest way to jump between micro-app something
              > that runs inside zope.
              
              Yes, and a ZCatalog index, if it turns out we want one, can always use
              a utility internally.
              
              Alec
              
        • Re: sprinters!

          from whit on Mar 10, 2007 08:22 AM
          >
          >> It might be cool to define the application-side API for an external
          >> search engine, and then implement it via each of the three backends.
          >> For one thing, that would let us test each one using identical 
          >> workloads.
          >
          > That would be nice.  I assume that would be a catalog that forwards 
          > requests and updates?
          >
          +1
          
          also... I believe kapil has some experience with pylucene and he will be 
          at the sprint. he should be on this list already too.
          
          -w
          
          -- 
          
          ------ d. whit morriss ------
          - senior engineer, opencore -
          - http://www.openplans.org  -
          - m: 415-710-8975           -
          
          "If you don't know where you are,   
           you don't know anything at all"                                             
          
          Dr. Edgar Spencer, Ph.D., 1995
          
          
          
    • Re: sprinters!

      from derek on Mar 09, 2007 12:29 PM
      I'm a relately recent refugee into the world of Plone from Java EE. I'm based at
      Georgia Tech.
      
      (One of) my long term goal(s) is to aggregate content into a Plone site using
      Atom and/or RSS, obtaining it from other Plone sites and other  non-Plone
      sources, re-skin it, and display it in a new structure as a different site. I
      know I can accomplish some of this with staging, but not all of it, and I want
      the whole enchilada.
      
      So, my short-term goal is a robust implementation of inbound syndication, PLIP
      128 Pluggable Plone Syndication, that can be included in the core of Plone 3.5
      and deployed as a third-party product on Plone 3.0 and maybe 2.5.x. See the plan
      here:
      
      http://www.openplans.org/projects/bbq-sprint/plip-128-pluggable-plone-syndication
      
      I'll be here in Chapel Hill through Tuesday and then will have to return to
      Atlanta. But I will be sprinting remotely on this and will be available on
      #bbq-sprint for anyone who is interested.
      
      For the next few days, I'll be in and out of #bbq-sprint as commitments allow
      and require. Feel free to email me directly if you think we have common
      interests.
      
      Quoting Ian Bicking <ianb@...>:
      > Oh, but more about me!  I'm interested in working on Deliverance
      > integration.  Deliverance is a theming intermediary -- it gets HTML
      > pages from somewhere (say, Zope or Plone, but could be anything) and
      > applies a "theme", i.e., the wrapper (layout, navigation, CSS, etc).
      > Right now we have some setups running Deliverance as an HTTP proxy --
      > which is useful, but adds another stage to setup.
      >
      > So, I'd like to figure out how people can use Deliverance in Zope and/or
      > Plone with much less setup.  I'll also show people how Deliverance
      > works, some about why it works the way it does, as we try to figure out
      > the best strategy for deployment.  Help from someone with knowledge of
      > Zope's innards would be helpful, specifically how to do transformations
      > without entirely pulling apart Zope.
      >
      > Deliverance: http://openplans.org/projects/deliverance
      
      Ian, this Deliverance thing seems to fit into the long-term goal I described
      above. I love the example on the Deliverance introduction page showing content
      in OpenPlans and then the same content themed completely differently for another
      site. I think there are big benefits from putting outbound+inbound syndication
      together with Deliverance. If you agree, then we should talk!
      
      Derek