ejabberd development
  1. ejabberd development
  2. EJAB-923

Publishing items with the same id counts towards max_items limits

    Details

      Description

      I'm having a strange problem with the pubsub system that I think is a bug. I created a new node with max_items set to 20 which allows me to publish 20 items to the node before I start to receive retract notifications when I attempt to add any more. This much works as I expect.

      The problem comes when I publish multiple items with the same id. Each individual publish counts toward my max items limit, even though publishing with the same id does not actually create a new item in the node. This means that I am only actually able to store the last 20 publishes, not actually the last 20 items (ie if I published 15 new items and then republished the last one 5 times, on the next publish my first item would be retracted).

      I have just finished building a pubsub system where there will be lots of edits (publishing again with the same id), and this problem is pretty major because it prevents users from storing the max number of items in the node.

        Activity

        Hide
        Christophe Romain
        added a comment -

        the publish item function should not add counter when publishing with
        same itemid.
        the list of new items is built that way
        Items = [ItemId | GenState#pubsub_state.items--[ItemId]]
        so if ItemId is a previously published itemid, you should just get Items
        as a list of same size as before publish.
        The size of that list is checked, and we keep the first N ItemIds.

        so i do not understand your point. which version of ejabberd are you
        using ?

        Show
        Christophe Romain
        added a comment - the publish item function should not add counter when publishing with same itemid. the list of new items is built that way Items = [ItemId | GenState#pubsub_state.items-- [ItemId] ] so if ItemId is a previously published itemid, you should just get Items as a list of same size as before publish. The size of that list is checked, and we keep the first N ItemIds. so i do not understand your point. which version of ejabberd are you using ?
        Hide
        Matthew Baron
        added a comment -

        Thanks, Christophe. I am running a trunk version of ejabberd from about 4 weeks ago. The problem I'm seeing is that after I do more than max_items publishes, I start receiving retract notifications for the first items i published, even if some of the publishes were to the same ids. This makes it so that I can't truly store max_items.

        I am going to work out a simple test case today.

        Matt

        Show
        Matthew Baron
        added a comment - Thanks, Christophe. I am running a trunk version of ejabberd from about 4 weeks ago. The problem I'm seeing is that after I do more than max_items publishes, I start receiving retract notifications for the first items i published, even if some of the publishes were to the same ids. This makes it so that I can't truly store max_items. I am going to work out a simple test case today. Matt
        Hide
        Christophe Romain
        added a comment -

        ok, 'im waiting your test case, and will investigate on that point.

        Show
        Christophe Romain
        added a comment - ok, 'im waiting your test case, and will investigate on that point.
        Hide
        Matthew Baron
        added a comment - - edited

        Ok. I have just reproducted this with a simple test case:

        1. Create a node:

        <iq type="set" to="server" id="create1">
        <pubsub xmlns="http://jabber.org/protocol/pubsub">
        <create node="/path/to/node"/>
        <configure>
        <x xmlns="jabber:x:data" type="submit">
        <field var="FORM_TYPE" type="hidden">
        <value>http://jabber.org/protocol/pubsub#node_config</value>
        </field>
        <field var="pubsub#node_type"><value>collection</value></field>
        <field var="pubsub#deliver_notifications"><value>1</value></field>
        <field var="pubsub#deliver_payloads"><value>1</value></field>
        <field var="pubsub#persist_items"><value>1</value></field>
        <field var="pubsub#max_items"><value>20</value></field>
        <field var="pubsub#access_model"><value>open</value></field>
        <field var="pubsub#publish_model"><value>open</value></field>
        </x>
        </configure>
        </pubsub>
        </iq>

        2. Publish the first item to the node:

        <iq type='set' to='pubsub.server' id='publish1'>
        <pubsub xmlns='http://jabber.org/protocol/pubsub'>
        <publish node='/path/to/node'>
        <item id="1">
        <name>First item</name>
        </item>
        </publish>
        </pubsub>
        </iq>

        3. Publish a second item to the node 20 times (so total number of publishes is max_items + 1). Since id is the same every time, it should overwrite and not count towards max_items.

        <iq type='set' to='pubsub.server' id='publish1'>
        <pubsub xmlns='http://jabber.org/protocol/pubsub'>
        <publish node='/path/to/node'>
        <item id="2">
        <name>Second item</name>
        </item>
        </publish>
        </pubsub>
        </iq>

        At this point, the first item has disappeared from the node.

        How can I double check what version of ejabberd is running?

        Matt

        Show
        Matthew Baron
        added a comment - - edited Ok. I have just reproducted this with a simple test case: 1. Create a node: <iq type="set" to="server" id="create1"> <pubsub xmlns="http://jabber.org/protocol/pubsub"> <create node="/path/to/node"/> <configure> <x xmlns="jabber:x:data" type="submit"> <field var="FORM_TYPE" type="hidden"> <value> http://jabber.org/protocol/pubsub#node_config </value> </field> <field var="pubsub#node_type"><value>collection</value></field> <field var="pubsub#deliver_notifications"><value>1</value></field> <field var="pubsub#deliver_payloads"><value>1</value></field> <field var="pubsub#persist_items"><value>1</value></field> <field var="pubsub#max_items"><value>20</value></field> <field var="pubsub#access_model"><value>open</value></field> <field var="pubsub#publish_model"><value>open</value></field> </x> </configure> </pubsub> </iq> 2. Publish the first item to the node: <iq type='set' to='pubsub.server' id='publish1'> <pubsub xmlns='http://jabber.org/protocol/pubsub'> <publish node='/path/to/node'> <item id="1"> <name>First item</name> </item> </publish> </pubsub> </iq> 3. Publish a second item to the node 20 times (so total number of publishes is max_items + 1). Since id is the same every time, it should overwrite and not count towards max_items. <iq type='set' to='pubsub.server' id='publish1'> <pubsub xmlns='http://jabber.org/protocol/pubsub'> <publish node='/path/to/node'> <item id="2"> <name>Second item</name> </item> </publish> </pubsub> </iq> At this point, the first item has disappeared from the node. How can I double check what version of ejabberd is running? Matt
        Hide
        Matthew Baron
        added a comment -

        Was this reproducible on your end?

        Show
        Matthew Baron
        added a comment - Was this reproducible on your end?
        Hide
        Christophe Romain
        added a comment -

        i'm sorry i can not reproduce your error
        i'm using last svn r2048 from trunk.
        i also made the test using ejabberd 2.0.5.
        and that bloc of code has not changed for a while, i don't understand what you see.
        from where did you installed ejabberd ? sources, package, installer ?

        Show
        Christophe Romain
        added a comment - i'm sorry i can not reproduce your error i'm using last svn r2048 from trunk. i also made the test using ejabberd 2.0.5. and that bloc of code has not changed for a while, i don't understand what you see. from where did you installed ejabberd ? sources, package, installer ?
        Hide
        Matthew Baron
        added a comment -

        Turns out I had two versions on my machine and I was actually running 2.0.3 while I was experiencing the issue. I am now running the latest version of trunk and I have no issues.

        Show
        Matthew Baron
        added a comment - Turns out I had two versions on my machine and I was actually running 2.0.3 while I was experiencing the issue. I am now running the latest version of trunk and I have no issues.
        Christophe Romain
        made changes -
        Field Original Value New Value
        Status Open [ 1 ] Resolved [ 5 ]
        Fix Version/s ejabberd 2.1.0 [ 10140 ]
        Fix Version/s ejabberd 3.0.0-alpha [ 10240 ]
        Fix Version/s ejabberd 2.0.5 [ 10301 ]
        Resolution Fixed [ 1 ]
        Christophe Romain
        made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Mickaël Rémond
        made changes -
        Workflow development v3 [ 68658 ] Development v4 [ 81153 ]

          People

          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:
              Days since last comment:
              4 years, 50 weeks, 2 days ago

              Issue deployment