Uploaded image for project: 'ejabberd development'
  1. ejabberd development
  2. EJAB-923

Publishing items with the same id counts towards max_items limits

    Details

      Description

      I'm having a strange problem with the pubsub system that I think is a bug. I created a new node with max_items set to 20 which allows me to publish 20 items to the node before I start to receive retract notifications when I attempt to add any more. This much works as I expect.

      The problem comes when I publish multiple items with the same id. Each individual publish counts toward my max items limit, even though publishing with the same id does not actually create a new item in the node. This means that I am only actually able to store the last 20 publishes, not actually the last 20 items (ie if I published 15 new items and then republished the last one 5 times, on the next publish my first item would be retracted).

      I have just finished building a pubsub system where there will be lots of edits (publishing again with the same id), and this problem is pretty major because it prevents users from storing the max number of items in the node.

        Expenses

          Activity

          mgbaron Matthew Baron created issue -
          Hide
          cromain@process-one.net Christophe Romain added a comment -

          the publish item function should not add counter when publishing with
          same itemid.
          the list of new items is built that way
          Items = [ItemId | GenState#pubsub_state.items--[ItemId]]
          so if ItemId is a previously published itemid, you should just get Items
          as a list of same size as before publish.
          The size of that list is checked, and we keep the first N ItemIds.

          so i do not understand your point. which version of ejabberd are you
          using ?

          Show
          cromain@process-one.net Christophe Romain added a comment - the publish item function should not add counter when publishing with same itemid. the list of new items is built that way Items = [ItemId | GenState#pubsub_state.items-- [ItemId] ] so if ItemId is a previously published itemid, you should just get Items as a list of same size as before publish. The size of that list is checked, and we keep the first N ItemIds. so i do not understand your point. which version of ejabberd are you using ?
          Hide
          mgbaron Matthew Baron added a comment -

          Thanks, Christophe. I am running a trunk version of ejabberd from about 4 weeks ago. The problem I'm seeing is that after I do more than max_items publishes, I start receiving retract notifications for the first items i published, even if some of the publishes were to the same ids. This makes it so that I can't truly store max_items.

          I am going to work out a simple test case today.

          Matt

          Show
          mgbaron Matthew Baron added a comment - Thanks, Christophe. I am running a trunk version of ejabberd from about 4 weeks ago. The problem I'm seeing is that after I do more than max_items publishes, I start receiving retract notifications for the first items i published, even if some of the publishes were to the same ids. This makes it so that I can't truly store max_items. I am going to work out a simple test case today. Matt
          Hide
          cromain@process-one.net Christophe Romain added a comment -

          ok, 'im waiting your test case, and will investigate on that point.

          Show
          cromain@process-one.net Christophe Romain added a comment - ok, 'im waiting your test case, and will investigate on that point.
          Hide
          mgbaron Matthew Baron added a comment - - edited

          Ok. I have just reproducted this with a simple test case:

          1. Create a node:

          <iq type="set" to="server" id="create1">
          <pubsub xmlns="http://jabber.org/protocol/pubsub">
          <create node="/path/to/node"/>
          <configure>
          <x xmlns="jabber:x:data" type="submit">
          <field var="FORM_TYPE" type="hidden">
          <value>http://jabber.org/protocol/pubsub#node_config</value>
          </field>
          <field var="pubsub#node_type"><value>collection</value></field>
          <field var="pubsub#deliver_notifications"><value>1</value></field>
          <field var="pubsub#deliver_payloads"><value>1</value></field>
          <field var="pubsub#persist_items"><value>1</value></field>
          <field var="pubsub#max_items"><value>20</value></field>
          <field var="pubsub#access_model"><value>open</value></field>
          <field var="pubsub#publish_model"><value>open</value></field>
          </x>
          </configure>
          </pubsub>
          </iq>

          2. Publish the first item to the node:

          <iq type='set' to='pubsub.server' id='publish1'>
          <pubsub xmlns='http://jabber.org/protocol/pubsub'>
          <publish node='/path/to/node'>
          <item id="1">
          <name>First item</name>
          </item>
          </publish>
          </pubsub>
          </iq>

          3. Publish a second item to the node 20 times (so total number of publishes is max_items + 1). Since id is the same every time, it should overwrite and not count towards max_items.

          <iq type='set' to='pubsub.server' id='publish1'>
          <pubsub xmlns='http://jabber.org/protocol/pubsub'>
          <publish node='/path/to/node'>
          <item id="2">
          <name>Second item</name>
          </item>
          </publish>
          </pubsub>
          </iq>

          At this point, the first item has disappeared from the node.

          How can I double check what version of ejabberd is running?

          Matt

          Show
          mgbaron Matthew Baron added a comment - - edited Ok. I have just reproducted this with a simple test case: 1. Create a node: <iq type="set" to="server" id="create1"> <pubsub xmlns="http://jabber.org/protocol/pubsub"> <create node="/path/to/node"/> <configure> <x xmlns="jabber:x:data" type="submit"> <field var="FORM_TYPE" type="hidden"> <value> http://jabber.org/protocol/pubsub#node_config </value> </field> <field var="pubsub#node_type"><value>collection</value></field> <field var="pubsub#deliver_notifications"><value>1</value></field> <field var="pubsub#deliver_payloads"><value>1</value></field> <field var="pubsub#persist_items"><value>1</value></field> <field var="pubsub#max_items"><value>20</value></field> <field var="pubsub#access_model"><value>open</value></field> <field var="pubsub#publish_model"><value>open</value></field> </x> </configure> </pubsub> </iq> 2. Publish the first item to the node: <iq type='set' to='pubsub.server' id='publish1'> <pubsub xmlns='http://jabber.org/protocol/pubsub'> <publish node='/path/to/node'> <item id="1"> <name>First item</name> </item> </publish> </pubsub> </iq> 3. Publish a second item to the node 20 times (so total number of publishes is max_items + 1). Since id is the same every time, it should overwrite and not count towards max_items. <iq type='set' to='pubsub.server' id='publish1'> <pubsub xmlns='http://jabber.org/protocol/pubsub'> <publish node='/path/to/node'> <item id="2"> <name>Second item</name> </item> </publish> </pubsub> </iq> At this point, the first item has disappeared from the node. How can I double check what version of ejabberd is running? Matt
          Hide
          mgbaron Matthew Baron added a comment -

          Was this reproducible on your end?

          Show
          mgbaron Matthew Baron added a comment - Was this reproducible on your end?
          Hide
          cromain@process-one.net Christophe Romain added a comment -

          i'm sorry i can not reproduce your error
          i'm using last svn r2048 from trunk.
          i also made the test using ejabberd 2.0.5.
          and that bloc of code has not changed for a while, i don't understand what you see.
          from where did you installed ejabberd ? sources, package, installer ?

          Show
          cromain@process-one.net Christophe Romain added a comment - i'm sorry i can not reproduce your error i'm using last svn r2048 from trunk. i also made the test using ejabberd 2.0.5. and that bloc of code has not changed for a while, i don't understand what you see. from where did you installed ejabberd ? sources, package, installer ?
          Hide
          mgbaron Matthew Baron added a comment -

          Turns out I had two versions on my machine and I was actually running 2.0.3 while I was experiencing the issue. I am now running the latest version of trunk and I have no issues.

          Show
          mgbaron Matthew Baron added a comment - Turns out I had two versions on my machine and I was actually running 2.0.3 while I was experiencing the issue. I am now running the latest version of trunk and I have no issues.
          cromain@process-one.net Christophe Romain made changes -
          Field Original Value New Value
          Status Open [ 1 ] Resolved [ 5 ]
          Fix Version/s ejabberd 2.1.0 [ 10140 ]
          Fix Version/s ejabberd 3.0.0-alpha [ 10240 ]
          Fix Version/s ejabberd 2.0.5 [ 10301 ]
          Resolution Fixed [ 1 ]
          cromain@process-one.net Christophe Romain made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          mremond@process-one.net Mickaël Rémond made changes -
          Workflow development v3 [ 68658 ] Development v4 [ 81153 ]

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development