Thursday, April 24, 2014

MongoDB scaling setup using AWS

~/tasks.txt.html

Using Python, Django, django-nonrel
Separating Read/Write 
---------------------
Achieving read/write separation using django-nonrel
    Add the additional database with slave_ok function
    ref: http://django-mongodb-engine.readthedocs.org/en/latest/reference/settings.html

Add database_routers:
    Follow the MasterSlaveRouter
    https://docs.djangoproject.com/en/dev/topics/db/multi-db/




Scaling Mongo Cluster
----------------------

Scaling by EBS volumes.

Use RAID10. The performance of RAID5 is not ideal
for MongoDB. RAID0 performs well but does not
provide sufficient fault tolerance. RAID10 is the best
compromise for performance and fault tolerance.

Highly Recommended to use multiple disks

Performance Suggestions:
NOTE: Costs are calculated base on 500GB of EBS storage volume for
/data on mongo with 1000 iops each

1. Minimum production scale (estimated monthly cost:  USD 667.56)


      MASTER                SLAVE
    ----------             ----------
   | m3.large |  -------- | m3.large |
    ----------             ----------
        |                       |
       ---                      ---
      |   | EBS                |   | EBS
      |   | PIOPS              |   | PIOPS
       ---                      ---


2. Medium production scale
Move up to higher bandwidth instance types (m1.xlarge, c1.xlarge, or
m2.4xlarge), and increase the number of volumes.
(estimated monthly cost on m3.xlarge USD 2163.73)

           --------------------
          |  MongoDB Instance  |
           --------------------
            |     |     |    |
           ---   ---   ---   ---
          |   | |   | |   | |   |
          |   | |   | |   | |   |
           ---   ---   ---   ---
           EBS
           PIOPS - RAID 10

3. Extra-large production scale (setup taken from postgres whitepaper)
Amazon EBS; CC2 and CR1 instances would make excellent primary nodes,
particularly when paired with a large number of Amazon EBS volumes (8 or more).
Amazon EBS with Provisioned IOPS volumes in a RAID set can support very
I/O-intensive applications.
(Assuming we have a secondary with the same instance.
estimated monthly cost on C3.4xlarge Estimated cost: $3738. 93)



       -----------------------------------------------
      |            MongoDB Instance                   |
       -----------------------------------------------
        |     |     |     |     |     |     |     |
       ---   ---   ---   ---   ---   ---   ---   ---
      |   | |   | |   | |   | |   | |   | |   | |   |
      |   | |   | |   | |   | |   | |   | |   | |   |
       ---   ---   ---   ---   ---   ---   ---   ---
       EBS
       PIOPS - RAID 10

4. Cluster large production scale
MongoDB recommendation for high scalability.
Estimated monthly cost base on c3.8xlarge, master shard only:
USD 6286.36



                   --------------------
                  | MONGO MASTER SHARD |
                   --------------------
                        /          \
       --------------------     --------------------
      |  MongoDB Instance  |   |  MongoDB Instance  |
       --------------------     --------------------
        |     |     |    |       |     |     |    |
       ---   ---   ---   ---    ---   ---   ---   ---
      |   | |   | |   | |   |  |   | |   | |   | |   |
      |   | |   | |   | |   |  |   | |   | |   | |   |
       ---   ---   ---   ---    ---   ---   ---   ---
       EBS                      EBS
       PIOPS - RAID 10          PIOPS - RAID 10


Sources:
http://info.mongodb.com/rs/mongodb/images/10gen-MongoDB_Operations_Best_Practices.pdf
http://media.amazonwebservices.com/AWS_RDBMS_PostgreSQL.pdf

No comments:

Post a Comment