Wednesday, February 9, 2011

what us Apache ActiveMQ

  • HIGHLY configurable. You can probably do anything you want it to with it
  • You can choose a message store. 4 are already available
  • Has lots of clustering options:
    • Shared nothing Master-Slave: ACK sent to client when master stores the message
    • Shared Database: Acquires a lock on the DB when any instance tries to access the DB
    • Shared Filesystem: Locks a file when accessing the FS. Issues when using NFS with file-locking; or basically any network based file system since file locking is generally buggy in network file systems
  • Network of brokers: This is an option that allows a lot of flexibility. However, it seems to be a very problematic/buggy way of doing things since people face a lot of issues with this configuration
  • Scaling:
    • A. Default transport is blocking I/O with a thread per connection. Can be changed to use nio
    • Horizontal scaling: Though they mention this, the way to achieve this is by using a network of brokers
    • Patitioning: We all know Mr. Partitioning, don’t we. The client decides where to route packets and hence must maintain multiple open connections to different brokers
  • Allows producer flow-control!!
  • Has issues wrt lost/duplicate messages, but there is an active community that fixes these issues
  • Active MQ crashes fairly frequently, at least once per month, and is rather slow - http://stackoverflow.com/questions/957507/lightweight-persistent-message-queue-for-linux
  • Seems to have bindings in many languages(just like RabbitMQ)
  • Has lots of tools built around it 12. JMS compliant; supports XA transactions: http://activemq.apache.org/how-do-transactions-work.html
  • Less performant as compared to RabbitMQ
  • We were able to perform some tests on Apache Active MQ today, and here are the results:
    • Non persistent mode: 5k messages/sec
    • Persistent mode: 22 messages/sec (yes that is correct)
  • There are multiple persisters that can be configured with ActiveMQ, so we are planning to run another set of tests with MySQL and file as the persisters. However, the current default (KahaDB) is said to be more scalable (and offers faster recoverability) as compared to the older default(file/AMQ Message Store: http://activemq.apache.org/amq-message-store.html).
  • The numbers are fair. Others on the net have observed similar results: http://www.mostly-useless.com/blog/2007/12/27/playing-with-activemq/
  • With MySQL, I get a throughput of 8 messages/sec. What is surprising is that it is possible to achieve much better results using MySQL but ActiveMQ uses the table quite unwisely.
  • ActiveMQ created the tables as InnoDB instead of MyISAM even though it doesn’t seem to be using any of the InnoDB features.
  • I tried changing the tables to MyISAM, but it didn’t help much. The messages table structure has 4 indexes !! Insert takes a lot of time because MySQL needs to update 4 indexes on every insert. That sort of kills performance. However, I don’t know if performance should be affected for small (< 1000) messages in the table. Either ways, this structure won’t scale to millions of messages since everyone will block on this one table.
Posted by at