3D Maps:
Practice Makes Perfect

Today, I had to work on a lot of 3D maps using population, population growth, and population density. I think I spent roughly around 6 hours today just working on the 3D maps for my client. Last Saturday was my first time to handle 3D maps and I think I have gained some nice progress in familiarizing myself with the basics.

Here’s my top pick for the day, because the colors I used here are UP’s logo colors. The green ones indicate a positive (increase) growth rate in the population of the barangay. The red ones indicate a negative (decrease) growth rate in the barangay. I guess there are a few pro-RH barangays if based on the data from 2000 to 2010, right? This is NCR data, which is tilted a little to show off its finest angles:

NCR Only

The one thing I enjoyed about making 3D maps with ArcScene 10 is the ability to rotate in various directions:

3D_PopulationGrowth_NCRProminent_darkened_frontview 3D_PopulationGrowth_colorcoded_midday_backview

Later on, I stepped out of the barangay units and proceeded to aggregate the parameters at the municipal or city level, where everything had an upward trend.

3D_Population_Muni_dividedby100_darken_backview 3D_PopGrowth_dark_backviewl 3D_PopDen_Muni_darken_backview

When I have recovered my necessary hours of sleep, I will post more about techniques in shadow and light rendering that I have discovered this morning. For now, I leave this visual treat

Murphy’s Law

Without going into necessary details, I must say that Murphy’s Law does happen. And perhaps this is why I am running on caffeine at 3am here at the office, thinking of the next task on this very long list. The worst possible things that a working girl can have for the day’s work happened to me this morning and it cascaded into the rest of the day. Still, I am thankful for the fact that the services of this poor and moody soul is highly in demand and this has caused a lot of the events that I did not imagine happening.

I won’t be posting a long one today because I am on “power saving” mode for the work day by 8am in a faraway place of assignment. Now, I am hopefully still productive in reading some books on Flood management. My poor brain can use 8 hours of sleep in a soft and air-conditioned bed. I promise to treat myself after this week is over. I really promise. <3

I have to really make amends to myself for this kind of self-imposed torture.

 

Dipping My Fingers
in Web GIS

Yesterday was the day for learning how to make basic 3D maps. Today marks the day for dabbling with Web GIS interface.

It’s been a very productive mapping weekend for me. And the most awesome part of it was that I did not have my bosses with me when I made these. I had the tendency to often ask their advice when I render some maps during weekdays. I had this weekend to somehow “play” and explore with what I have learned so far and ended up uncovering new features. I have learned to trust myself again after a few weeks of feeling like not making it in this chosen path I had for myself in my career.

For starters, I went for the simplest web GIS interface and here is what I came up with as a demo or basic data. I like the blue marble MapBox layer which served as my basemap in this image below. It’s a bathymetry basemap although I also tried using OpenStreet Map and Google Satellite/Hybrid tiles.

It operates perfectly in Mozilla Firefox, subject to the nuances of one’s internet connection.

blue marble

I like the fact that I began this journey and blogged about my experiences because I was able to track much of the things I have learned over the last few months. And I am amazed because last year, I had nothing to blog about that is of some value to people or peers in my current industry or line of practice except my sad feelings. Hohoho.

So far, I am able to load this in a local host with basic querying and editing functions. It’s really very flexible and modular in form, making it easy to learn or self-study. But I find that I am still not able to dig deeper in the foundations of doing this things, so I still need to study this technology further.

I know that this is not going to be the last time I am going to write about mapping-programming adventures. 🙂

[I still have one more task on my plate today and I hope that I still have the mental energy for one more productive venture. :)]

 

 

Thoughts
on Expertise,
Web GIS,
and Open Source

Ever been subjected to that awkward moment where someone was dying to impress you about how superbly expert he is about something but he ended up doing the total opposite? Around five months ago, somebody messaged me on Facebook about my Facebook status/announcement for a potential programming-mapping project. He said that he can just work for my company as a moonlighting programmer to customize the utility app, because it was “so simple” and he was bored with having mastered his job already.

Wapak!

Four words flashed in my head that instant: Ang yabang naman nito!!!

(Two other mapping guys a hundred times better than him with skills and programming prowess inquired about the project respectfully, and became our company’s partners beyond this initial project.)

I actually thought highly of him before this incident. When I told him that my client is using a customized app that does not use his tool of choice, ArcGIS 10, he immediately withdrew his offer and said that he cannot do it given that it’s not going to be implemented through the software that he knows like the back of his hand.

His arrogance has been extinguished almost instantly and he suddenly typed in “GTG,” that he’s busy with managerial stuff for his work, as if I was wasting his time. I wanted to call him up and LOL him like crazy to drown out the awkwardness. Instead, I said “OK, good luck with your managerial stuff!”

Today, five months later, he is an introduction to a long blog post about expertise and mapping thoughts.

This tiny event led me to ponder along with other ideas for five months: How can one be an authentic expert in something if you do not probe in the depths, if you do not have the thirst to work from the ground up and see what you find? How can you call yourself an expert programmer if you do not have the versatility to work from the very foundations of what you do?

(And he had the temerity to market himself as an expert in this industry. I feel embarrassed for the true experts who don’t concern themselves with such things just to get attention from other people.)

I do not know if this idea is something that comes across to some app users, but I have been picking my brain constantly regarding an issue of how ease of use can sometimes make end users too lazy for their own good. I cannot help but feel that ease of use in software, while it is marketed as a time saver and the logical way to go for solutions, actually trades our ability to empower ourselves and innovate on our own devices.

When you are given an easy to click interface or pre-packaged piece of application that you only need to pay for and upgrade annually, you live  with the precious cost and accept that you will never have full knowledge of how exactly the system works from ground up.

Some people are content with this “restricted” kind of knowledge and call it expertise.  But try changing the application and then these “experts” will no longer know how to do things because they’re stuck with a single way of doing things.

It makes them extremely inflexible and ignorant.  True think tanks of this planet are not afraid to work with the barest bones to build an entire body or system even if it takes an entire lifetime’s work.  True think tanks tinker with their boundaries and are versatile.

And most all, think tanks don’t brag then cower in shame if their tool of choice is not available on the table. They rise to the occasion and find solutions with dogged persistence.

I find that when I overuse easy to install and easy to use apps, I get somewhat “dumber” although I accomplish things that may astound an unsuspecting passer-by. I get to appreciate the shortcuts more when I know how it works in the background, if I can see the “skeleton” of the main thing being sold, if I can take things apart and understand why this certain function crashed, and fully understand why this particular command did not generate the results that I want.

I do not know why, but this is how I learn that I have truly learned. And this is the standard by which I hold people who call themselves intelligent or good at what they do. And based on this standard, I am a total beginner to mapping and programming. I love it well enough to post every considerable thing I can write about it, but I still have a LONG WAY to go.

In the interests of saving time, we occasionally delude ourselves into thinking that the expensive and user friendly software empowers us to do more for less. In fact, it makes you dependent to a brand to sustain your skill set.

Living in the third world country, I find that the growing community of open source GIS developers as a resourceful and cost-efficient alternative to branded moguls is a very promising one. I am not yet as good as the least of them, but I support them and I want to be like them.

Long live Open Source. Programming is king.

Visions

(image credits to Engr. Ranel O. Padon)

It’s like when I personally discovered that I can style my clothes without succumbing to brands, that a Php 10,000 get-up can be reduced to Php 1,000 if you open your mind for saving the little wads in your wallet. It’s really a matter of perspective and willingness to be open to new ideas that make the most revolutionary ideas.

Despite not blooming fully as a seasoned person in the practice, I have realized that I can be an excellent GIS specialist without being impoverished with maintenance costs of premium software. To be fair, branded GIS software does provide the ease of use compared to open source.

But if you are interested enough in GIS to give it your time and working attention, you can take the extra time to make the free version work for you  as well as or even sometimes better than the paid version that people have marketed for the longest time without challenge.

To balance, I still use expensive software but I try my best to step out of the shortcuts and understand the principles behind the shortcuts that the software provides. It makes life more meaningful that way as a worker.

I know that I have posted a lot about GIS software lately, particularly ArcGIS 10. I have come to respect the very easy user interface provided by ArcGIS 10 because it helps me meet my deadlines, but like what I mentioned earlier, I am also open to other types of powerful software particularly the open source ones. The open source ones somehow unveil the heart and soul of what GIS can do.

This is what makes learning so versatile. There is so much to explore here. When I was in college, I was only dipping my toes at the tip of a very deep pond. I was dabbling with population data with simulated real-life problems. At work, I solve real engineering and mapping problems and it gives you a sense of purpose when you are not producing outputs just to get a medal for your bedroom wall.

The deep pond, as I learned from my first year in the working world of engineers, was in fact a vast ocean of knowledge that needs flippers and bunch of other stuff. You find coral reefs of analysed geographic information bits, oil spills in the form of messed up data, and many other things. In school, they train you in a pond with ideal conditions occasionally throwing the deadline shark here and there. At work, you are in the ocean with all those waves and it’s up to you the surf the tides and emerge victorious.  

The ocean in its vastness allows you to choose various vessels. So far, I have unveiled in my blog the use of the common vessel for GIS or mapping specialists like me: the GIS software route. I was quite inclined to write more about ArcGIS 10. Again, I promise to write about my QGIS experience some other time.

Despite the popularity, one clear disadvantage of relying on GIS software is the giant space requirement and the technical knowhow required to operate and maintain it. When you use GIS software, you also experience physical limitations like the video card requirements. For example, when I am doing mapping stuff on an i3 Pentium processor, I occasionally find it not responding or crashing during an intense geoprocessing activity when used with other apps like Skype and Mozilla Firefox.

In comes Google maps and they have promoted the concept of crowdsourcing, where even non-technical people (their Asia-Pacific bigwig once said “even an 11-year-old can use our system”) can contribute to a growing spatial database. But Google has a limited set of things they can share as open source and the rest has to be paid dearly with a premium.

Another apparent disadvantage is that when one technical staff member corrects a database entry from his computer, it is not as easily reflected in the computer of his other team member who might be using the same layer. I personally experienced this at work where my officemate has edited something on a road network file and I was using an obsolete version already. We don’t talk a lot because we are all busy so when you need to scramble for a quick thematic map for a client, you do not have the luxury of time to verify if you are using the same file. It’s cumbersome but it happens especially when you consider large institutions or agencies.

Sure, we can set up a server and a local network, but then you can have conflicting copies of the same file if they are accessed and edited at the same time by different people working in multiple computers.

Small wonder that these days, more and more people and institutions are now opening up to what is considered as the Web GIS segment of the mapping world. It’s Coke Light to the heavy Coke that is given by the usual mapping software. Even the premium mapping mogul ESRI accommodates this growing segment of the mapping world that makes use of internet maps that do not require an installation of or purchase of their ultra-expensive GIS software by having their own free web viewer.

Definitely, this is not the last time that I will write about working with the skeletons of GIS giant applications. It’s me thinking aloud about a subject that I am passionate about. I have finally warmed up to the idea that I can use various mapping vessels (ex. GIS software, Web GIS, programming, etc.) to achieve a common end, and whatever vessel I choose, it will make me a better mapping sailor if I am going to be very accommodating and fearless in the face of uncharted waters.

Who knows, that just might be the road of making me the captain of my own GIS ship?

Or perhaps I can be content with canoeing or a boat or a few columns of a sun-kissed raft. The possibilities are endless. At the moment, I am scoping the area and sunbathing with my BB cream.

I have found by experience that the really kick-ass and worship-worthy GIS navigators would spend time digging deep in the cool finds of their respective GIS vessels than idly send out self-glorifying Facebook messages. And for me, they’re the true programming heroes worth liking and befriending on Facebook.

 

[Special thanks to Engr. Ranel O. Padon for being an inspiring programming teacher and for providing me with my first GIS wooden raft. 🙂 ]

 

Making 3D Maps
Using ArcScene 10

Today, I consider this as a milestone at work and in GIS practice because I was finally able to construct my first set of 3D maps.

The Original Problem: Produce a map of population values at barangay level of Metro Manila and 4 adjacant provinces. Use bar chart rendered in 3D to show the values.

Using my ever trustworthy ArcGIS 10, here was my original bar chart map for the barangay level of NCR and its four adjacent provinces:

BarChart_Population_2D

As you can see, it’s not so pretty, right? I mean, some of the population values are so small and you can hardly see the distinctions among the boundaries. And then it looks cluttered and flat. And DULL.

I was using ArcGIS 10’s ArcMap 10 when I first handled this particular map request from my client. The bar charts are already in 3D but definitely, we know that it will look much better if we tilt the basemap a little vertically so that we can see the bars going upwards.

I did realize that 3D mapping is not entirely possible in ArcMap at its fullest functionality. There’s 3D Analyst toolbar but then the most I can do was simply to rotate the map from the top view using the checkbox located in the Properties section of the layer being rotated. It was still top view.

The next thing I did was purely exploratory. For the record, I had a client waiting for my output this evening and even though I was pressed for time, I tried my best to find the right set of commands to execute the required map.

Google helped a lot since I was also checking out websites like Stack Exchange and ArcGIS Resource Center.

I closed the ArcMap program then I used ArcScene 10 for the remainder of this exercise. I just reloaded the layers I used in ArcMap in this new map at ArcScene. It helped that the interface was the same but the gear of ArcScene is to create 3D maps instead of the usual 2D ones.

Using the ArcToolbox for Feature to 3D Attribute, I succeeded in converting my original 2-dimensional layer basemap into a 3D one.

But it still did not have the effect that I wanted for my client.

Here’s the top view after my feature to 3D attribute geoprocessing tool at ArcScene 10:

Screenshot (51)

When I tilted the view to reflect the side or profile view, I was aghast to find that the elevation values looked like snowflakes and was therefore unusable. I needed it to have tall and solid stacks, but instead it looked like this in reflecting the values of the population as Z-value:

Screenshot (52)

 

Fortunately, I had the urge to check out the properties of my layer and then I went to the Extrusion tab.

Extrusion tab turns points into lines, lines into polygons, and polygons into solid blocks. The polygon to solid block feature was precisely what I needed to have a solid stack at my disposal so I tinkered there and placed the appropriate expression:

Screenshot (53)

 

At first, I was unable to get the elevations right so it looked messed up in 3D wedges like this:

Screenshot (50)

 

Eventually, I managed to find the right combination of parameters in the Extrusion tab that allowed me to produce the 3D map that looked seamless:

3D_Population_Barchart

 

I also experimented on using a different color and a different perspective and here is the other snap shot of what I came up with.

3D_Population_Barchart_2

My client was finally happy. I am also happy because I am quite new to the practice and I never really imagined that today will be the day of learning how to make 3D maps. 🙂

I guess there is so much more to learn in the field of GIS. I am also in the recent process of learning more about Web GIS and I plan to write about that too in the coming days. GIS is a very nice area to explore; there are so many things that can be done with it and it keeps on growing dynamically. 🙂

Codecademy:
the Fun Programming School
for the Unschooled

Screenshot (33)

My chief passions are fairly few and far in between and they often involve computers. Imagine my giddiness when I discovered Codecademy. A friend told me before that bar none, W3 Schools is the best online website resource for learning all kinds of programming languages. I tried working my way up with W3 Schools but I found it a bit too dry for my taste. Months later, I stumbled upon Codecademy. And while I use W3 Schools as my online reference material for the HTML codes that I forget, I find myself hanging out at Codecademy portal not because I have to but because I want to.

It’s nice because I need to learn some PHP for new work responsibilities and the interface of Codecademy is interactive. I enjoy the ground up approach very much and it does not assume that you have prior background in programming. In college, I may have taken a couple of programming subjects but I did not have the chance to explore it. I got a flat 1.0 in C++ basics, but that’s pretty much it.

The only downside is that it just focuses more on technique. If you want full foundations and theory, you won’t find it in there. But if you just want to learn how to code and use it immediately for your web needs, I think it serves its purpose well for a FREE RESOURCE. Yep, FREE ITO. 🙂

Codecademy also has a loaded environment on the right side that allows you to see the results of what you code immediately. They have tracks for HTML, PHP, Ruby, Python, and Java Script. The interface and language used in this portal is so easy that even my 13-year-old sister can understand how it works.

I ruminated on enrolling for a Master’s degree to make myself seem smarter and accomplished like what everyone else within my age range seems to be doing. Given my constantly traveling and busy schedule coupled with intermittently long hours that spread from a few hours to a full overnight session, I don’t think I can afford to enroll in any physical class. Open university proves to be a bit too expensive at this point and I cannot juggle it with my upcoming wedding expenses. And besides, if you have something like Coursera classes already, Open University’s tuition fee offers the advantage of giving you a diploma and a curriculum.

You can still learn everything online if you have the time and a good computer with internet connection and lots and lots of EBOOKS. It’s like practicing engineering and not having a title and practicing engineering with a license. You’re doing the same thing, just that, the other one seems more prestigious when keeping up with appearances for people and expecting salary increase especially in academic institutions. 🙂

I have to admit that I may have the title engineer in my name but I know of non-engineers who are more well-versed in field work than I am. It really happens.I pay my respect to these people because they get paid so little and do so much in the construction industry. 🙂

Now, I am a licensed engineer veering a little closely to programming lately because of the Geomatics and IT wing of my multi-faceted degree. 🙂

At this practical age, I’m not that interested in pleasing people for as long as I am pleased with what I am learning, officially or unofficially. 🙂 For learning how to code, I can just learn just for the sheer pleasure of it and the possibility of tying it with all my mapping pursuits. Nothing beats self-motivation.

So if you’re like me,  more concerned with learning just for the learning part and not for the certificates you can show off on your bedroom wall or brag about to your friends just for the heck of it, you can try programming by Codecademy and look at many other online courses.

I really admire the brainchild of these programs because they are so generous in sharing themselves and changing lives. If I complete Codecademy’s tracks in a year, that’s tantamount to enrolling in around 6 subjects in a college or university. Isn’t that nice? I don’t think other people will be that genuinely generous especially if it does not benefit them personally. I am quite happy to have bumped into the generous side of the web, and yes, I plan to give back and spread word especially if it is well-deserved.

I am a fan of Open Source. Long live open source and free programs online. <3

Thanks Codecademy and more programming power to your organizers. You guys are beyond awesome.

 

 

 

 

 

 

 

 

(Almost) Digitizing Zen

One of the most mechanical procedures in the mapping practice is called digitizing. Some people already play with Google Earth and do not know that they are already digitizing when they create polylines to mark features like roads, rivers, and other linear features easily identifiable on the map. In Google Earth, you can save it as a part of a kmz or kml file.

The good thing about mapping is that when things fall apart, your map still holds itself together.

Going back to my main topic, lest I infect you with my morbid and depressive lethargy… There are numerous mapping software, and most advanced online geoportals also have that digitizing function it. You need not look far, even Google Earth has that.

What can I say? It’s absolutely absorbing but if you are not into it, it can get boring. I, for one, am feeling very much marooned these last three days. And I found such comfort (I know, geek comfort!) in digitizing everything away. It’s just like playing Candy Crush where you line things up in an array that makes sense. In the case of maps, you just line up the feature as you need it for your analysis.

Usually you will need a basemap or image (technically called raster) to be able to digitize. In ArcGIS 10, you have some maps to choose from:

Screenshot (22)

I’ll choose Open Street Map for this tutorial, since it has that cartoon-ey feel.

The next step involves determining what exactly is it that you want to digitize from the basemap. It can be a line or a polygon. Common line data includes rivers and roads. Polygons includes lots, buildings, blocks, and other imaginable feature on the map that is of interest to you.

I chose Paris as the geographic area of interest. It’s pretty chaotic, the road system in Paris. It’s made of these radial arrondissements. I distinctly remember in college when I took an urban planning subject and one of the books in the library showed a picture of the radial roads in Paris. Despite the artistic look, it served a very big logistic or transport issue. Maybe that’s why the velo or bicycle and walking is more encouraged than driving around this road network.

Screenshot (23)

First, you need a layer and then pinpoint that place you’d like to create a feature out of. In ArcGIS 10, you can go to ArcCatalog, create a new shapefile by right clicking on the white screen on the folder of your choice.

Screenshot (24)

That will prompt a new dialog box that you need to fill out accordingly. I already chose to outline gardens in a polygon shape, so:

Screenshot (25)

You can also choose to edit your coordinate system at this point. Usually I just import the coordinate system from another shapefile I have created previously. If you don’t have that, just find the appropriate coordinate system. If you are a fan of Google Earth, WGS84 is a very popular choice. You can also input manually or do whatever rocks your boat, basically:

Screenshot (27)

 

Back to the fun part, then. After clicking ok on that dialog box, you’d be able to exit Arc Catalog and go right back to the mapping software to identify that place you want to trace or digitize. In my case here, I chose Jardin des Tuileries, the famous garden for lovers in Paris. 🙂 I may never be able to go to Paris with my humble salary, but I think I can just be content looking at it from a geographical perspective and play with my maps until my final breathing day.

 

Screenshot (28)

 

Simply add the shapefile you formed from the previous step:

Screenshot (29)

Then click the Editor toolbar to start editing:

Screenshot (30)

After clicking that, a new sidebar appears on the right for ArcGIS 10 and you can choose your “weapon” of choice at the bottom. I choose Polygon, since I am targeting an octagonal region that won’t fit in a rectangle or circle:

Screenshot (31)

Once you click that, the cursor changes into a plus sign at the drawing area, and you can start clicking at corners, one corner at a time.

Screenshot (32)

After the last corner, double click, and you got yourself a polygon footprint for that shape. 🙂 You can turn off the original basemap you loaded earlier and you will still have the polygon. 🙂

It’s an absorbing hobby, making map features like this. Time flies when you digitize because it never ends, and you never run out of features to play with.

I only hate it when the task accumulates and I am stuck with a hundred roads in a rather redundant or non-exciting place. But honestly, when you want to forget or have a useful diversion, this is a pretty good activity. It won’t require you to think about your life or things if you are having an especially bad day or week. And then, you can just make it more artistic on your end if you want to. You can make the roads color purple or pink, in any style you fancy and in any number you please and you will not get condemned for what you choose. After all, it’s your rules and your style that governs a map that you make.

Shit only happens with your map when you have a horrible machine that hangs or does not have enough video card or RAM specs to boast for it. Anywhere else, it’s a frigging land mine, bad things can happen to you all the time and you cannot do anything about it. You can only control how you respond. Maybe that’s why I love maps so much.

They’re so neutral, so therapeutic, and much easier to handle than, say, people and life decisions. Can I just drown in these maps and fall into an eternal spell of geographical stimulation? Many people like going places, but I’m pretty happy sitting in one place if it means my head won’t run needlessly and if it means that I can just exist peacefully in my own bubble.

Digitizing is almost so Zen, if it were just the only thing in earth that had to matter.