This week we will talk about:
Check the class notes page to see what's coming up.
You all know what a SCRUM is by now, so we're going to take a few minutes once a week to do a SCRUM on your database projects. Pick the appropriate date for your section. Remember, the SCRUM format is:
Project groups are listed in the previous section. In the essence of time, we will not endeavor to be fair by making random people go first or last… it'll just go the way it is. Monday–Wednesday section will scrum on Wednesday. Tuesday–Thursday section will scrum on Thursday.
As mentioned elsewhere on these pages, Mongo is a document-oriented database, NOT a relational one,
which means it's part of the family of NoSQL
databases. In this type of database, the idea of what
we are used to as a row
in the traditional relational database is replaced by the concept of a much
more adaptable model known as a document
. This flexible architecture allows things like embedding
arrays, objects, other types of structures, binary document files, binary audio and video files, and many
more. Further, because of this approach, there is no need to pre-define a schema for your database. This
fact means that you can get the database running more quickly, and also implies that if you need to change
things [add a 'new column' to the table, etc.] there is very little difficulty and hence little effect on
the up-time of the database. You can also more easily experiment, and iteratively develop an application
that is optimized or tailored for your application.
The idea of scaling
your software involves being able to adjust the size of your application in a few
different ways. First of all, there is the idea of ecomony of scale, a term applied [mostly] in
the busienss world, meaining that because you have a large number of something the cost decreases so you can
make more profit. Think about Costco or WalMart, where they can buy in HUGE bulk from their suppliers, then
sell to their customers for a relatively small amount over their cost, which saves money for them, and also
saves money for their customers [which keeps them loyal as a side-benefit].
The other idea of scaling, as it applies to databases, is to be able to scale the application to adapt to growth and expansion. As the amount of data in the database increases [usually a good thing, and usually due to the success of the application] it eventually becomes necessary to expand the storage amount to handle the increase. This can be done in two ways: 1) scaling up, or 2) scaling out. Each of these has a benefit and a drawback. Scaling UP means buying a new computer that has more storage within the single box, which is often a substantial increase in the cost of the hardware; but, this single box is easier to administer since there is only one, it is entirely self-contained. Scaling OUT means buying multiple boxes, which provides flexibility for buying the same or improved hardware at a reduced cost, but incurs the difficulty of having to administer multiple boxes across a network. However, scaling out can also mean more easily expanding by adding nodes to the network, which then makes it easier to support widely-distributed backup and failover systems.
In some cases, of course, this can be a distinction without a difference.
Ultimately, the point is that Mongo is optimized for scaling OUT. Document-orientation is easier to scale
across multiple servers, since you don't have to split up tables and use fancy indexing schemes like the
trees we've studied to get faster response. Also, the Mongo DBMS [yes, it's still one of those!] handles
the load balancing
chores for multiple database machines, automatically re-distributing all the
documents in the database, and handles routing of user requests to the server containing the specific
data. Finally, Mongo can figure out automatically when a new server has been added and will reconfigure
things to handle the new storage.
The basic unit of storage in mongo is the document
. Documents [as previously stated] are roughly the
same as a relational row
in a table. The next level up is called a collection
, which would be
similar to a table
in the relational schema, but with a dynamic schema, meaning it can change
from table to table and even from document to document [row to row]. The next level is a database
,
which can have lots of collections. Each instance [installation of mongo] can have multiple databases, each
of which can have its own collections.
Within each document, there is a special id value, called the key
, which is unique within each of the
collections in a database. This allows you to uniquely identify any document in the collection, much as a
primary key is used to uniquely identify any row in a table in a relational schema. And finally, there is a
simple built-in JavaScript shell
, that is used primarily for administration.
A document in mongo is an ordered set of key/value pairs. That makes it REALLY simple to represent documents using JavaScript. Documents are actually stored using the BSON binary format, but that is totally transparent to us as database developers. We just care about getting information in and out, so we can use whataver the language we are using provides us with. For starters, we'll just use JavaScript objects, since that's something with which we are extremely familiar. So, for example, a simple document would have the following form:
{ "sayingHI" : "Hello, world!" }
As with any programming language,there are distince data types with mongo, and there are rules for which
characters can or cannot be used in key names. Keys may NOT contain the null character [\0
] since
that character sequence is used to signify the END OF A STRING, just like in the C-based languages. Also
the dot and dollar characters have special meanings and should be considered as reserved words or tokens
for reasons that are detailed in your text book. Mongo is also case-sensitive; you all know what that
means. Finally, and perhaps obviously to you as database designers, a document cannot contain duplicate
keys.
Documents can be grouped together into collections
, and there are several advantages to doing this
to organize things. One advantage is that mongo is optimized to be able to run on distributed servers;
part of this is the ability to be smart about breaking the data up so the queries still execute quickly.
Thus, grouping documents into collections facilitates mongo implementing data locality
, which is
partitioning related data on the same drive, partition, or server. Another thing that appropriate use
of collections will do is facilitate indexing. We've seen how indexing can speed database access. Yet
another idea that is helped by collection grouping is speeding up the queries by limiting the amount of
data that needs to be sifted through.
Mongo will also allow you to have multiple databases in a single installation. This is partly because of the way it is constructed, but also partly because of it's scale out flexibility. To see the databases that are contained, or to set the one that you want to use, you can simply use the name of a specific database. Having multiple databases on the same installation can be very useful when you have a small operation that is only running one mongo server, but have mutliple applications that require different separated data stores.
There are quite a few people who have strong opinions about mongo specifically and document store databases in general. A couple of things are available from Dr. Toal's database page on mongo, and the links are reproduced here for convenience. Let's take a look:
Here are a couple more people with things to say, which are informative:
|
First, install mongo. Download it from their
website. Make sure you get the correct version for your computer and operating system. Install it
using the package and method which is appropriate to your computer. Make a directory in which you can
play around, which is known as a sandbox
, and remember the directory name for the following start
command. You will probably need to make sure the program path is in your search path on windows boxes, and
there may be something similar on mac/linux/unix that you'll need to do in your bash profile. Once mongo is
installed, start a terminal window [or your application folder or your windows start button and just start it
from the O/S GUI] and start the server using the following command:
mongod --dbpath <whatever path you created> --fork --syslog
The --fork
switch on the command line will make the server run in the background. so you'll have to
know now to find it in a top
window, in a terminal
window, or in the task mangler
in
order to turn it off later. The mongo server runs on port 27017 by default. You can configure other ports
later.
BTW, starting the server up in this way is HIGHLY insecure, so DO NOT do it this way if you are creating a real mongo installation. Learn how to secure your server so that you don't get hacked, losing valuable information and probably the respect and confidence of your customers. In short, professional stunt driver on a closed track, don't try this at home.
Next start an instance of the client, using the mongo command:
mongo
Jeez, that was simple.
Now, try the following commands in the client window. Do them in order so that you can see what happens.
Graph Databases, exemplified by Neo4J