Comments on: Why not to use Excel for Data Gathering

By: Mike

Mike — Tue, 11 May 2010 13:02:59 +0000

@Ian Welsh: That is the problem. Most managers, once they get to the office/level they’re at, seem to lost all sense of what is right and wrong – in terms of tools to do the job.

By: Johan L

Johan L — Sun, 10 Aug 2008 22:35:30 +0000

Nice troll. Reality check

By: Neil

Neil — Sat, 09 Aug 2008 18:35:00 +0000

Nice, Mark.

The last time I actually wrote an Excel import script was for a manager who had already sent out several thousand questionnaires, and was starting to get them back. He couldn't figure out how to turn them into usable data.

I wrote him a script (VBS, monitored a folder, and ripped any XLS that was dropped into it, cleaned it up and shoved it into Access.) Told him how to set it up, and how he'd have to fix the spreadsheets if the script spit them back at him, and that if he detailed anywhere it was consistently breaking, we could update the script.

After that, he started asking for my opinion earlier in the decision tree.

By: Neil

Neil — Sat, 09 Aug 2008 18:25:15 +0000

Spune, if you have any database people where you work, get their support. Update queries are easier (ie, less money spent on labor) than updating spreadsheets.

By: Neil

Neil — Sat, 09 Aug 2008 18:22:22 +0000

Debra, I'm certainly not knocking Excel as a product. In fact, it's pretty handy as an output device (charts go over well with management) and it's easily automated.

The article was in response to a couple of projects that consisted of sending out a lot (literally tens of thousands) of spreadsheets, which would have a fair number of free-text fields) and then importing the data on an ongoing basis.

In this project, just the version overlap alone (ie, you come up with a glitch in the spreadsheet, you discover it will import better with a different value (or management changes something...); you still have thousands of spreadsheets floating around. There's no updgrade potential.) suits it for a centralized input device.

For smaller, simpler (and certainly quicker) jobs, Excel is low-overhead, and low learning-curve.

By: Neil

Neil — Sat, 09 Aug 2008 18:14:28 +0000

PC, I have nothing against Python...I haven't used it simply because it hasn't shown up as the right tool for the jobs I've done. (I will say, though, that flaming Perl isn't much of a reccomendation, in and of itself. How is Python in regards to parsing English text? I know it rocks at number-crunching. And how is Python for compiling into an EXE? That was one of the specific points.)

BTW, I'm also not a Perl developer. The email was about *not* doing this sort of project in Excel, not about any given language.

By: Neil

Neil — Sat, 09 Aug 2008 18:09:09 +0000

Interesting thought, Ibrahim. And I'll have to look into it.

It won't happen in the office, as it's a secure site. If infomration got posted on Google's servers, our security depeartment would have a well-earned cow

By: Kevin Merritt

Kevin Merritt — Sat, 09 Aug 2008 16:37:01 +0000

This is one of the primary use case scenarios that we are solving at blist. We have created an online database that allows mainstream users to work in an environment that looks and feels like Excel, but the columns are “typed” – meaning the creator of the table can control the type of content people enter. We have checkboxes, T/F booleans that can be customized (hire/no-hire; pass/fail; go/no-go), picklists, etc.
In addition to its ease of use, there’s a huge benefit downstream in that it really is a database. You can analyze, print, chart, export, etc.
We support import and export from and to Excel.
It’s free to use. Try it out at http://www.blist.com.

By: Mark

Mark — Sat, 09 Aug 2008 15:30:49 +0000

It’s a knock-off for a scripter, too. And then another knock-off. And another. And another. Hey, you’re right, pretty soon that is a career.
I once worked with someone who was in charge of managing all the data for a team, who seemed to have the job only because she was incapable of doing anything else. She did everything with Excel, and there problems everywhere. What a mess. When I needed some data collected and processed, I had to go through her, according to the rules (she inserted herself in the pipeline between data and build). So I built a web app, as you suggest, for my portion of the data collection, and had her use that. She never got to touch the data, only the web UI. Worked like a charm.

By: Spune

Spune — Sat, 09 Aug 2008 13:15:17 +0000

WOW, you hit the nail on the head. the excuse that we get alot at work is we can’t modify hundreds of entries at once thru the web like we can in an excel file. The problem is parsing as you’ve already stated and we still have to treat the whole excel sheet as a single transaction. Most of this data is coming from other systems, but each time I mention some kind of API to have those applications upload their data automatically I get stares of confusion, anger, or fear.