When transferring files from system to system, for instance, using csv files to transfer data from one db to another, sometimes there are problems where programs will not process a file because of its line endings. This happens especially if you process a file on one platform, say Mac, and try to use the file on another, say Windows. Even if you have saved a file as CSV from Mac Excel 2008, it will not necessarily be saved in a format that can be read programmatically, if the program is expecting a certain type of line ending.

How Can We Avoid Line Terminator Problems and Troubles?

Let's recall how lines are terminated by default on Windows, Mac and Unix.

• Windows-style line endings are CRLF ( \r\n or hex 0D0A )
• Mac-style line endings are CR ( \r or hex 0D )
• Unix-style line endings are LF ( \n or hex 0A )

There are a number of ready-made command line programs like unix2dos, dos2unix, mac2dos, dos2mac and so on, that can be used to convert line endings. Note that you can also use the tr or perl commands as well. Tr is available on Macs by default and on almost any unix. Perl is pretty ubiquitous as well. E.g:

[root@server /path/to/files]# tr '\r' '\r\n' win-crlf-file.csv

[root@server /path/to/files]# tr '\n' '\r\n' win-crlf-file.csv

[root@server /path/to/files]# perl -pe 's/\r\n|\n|\r/\r\n/g' unix-lf-file.csv > win-crlf-file.csv

If you want to find out whether a file has the expected line terminators, you can use the file command on *nix or Mac. Here's what that looks like:

[root@server /path/to/files]# file inputfile1.csv

inputfile1.csv: ASCII text, with CRLF line terminators

[root@server /path/to/files]# file inputfile2.csv

inputfile2.csv: ASCII text, with CR line terminators

You can also use the cat command to show line endings, with its -e switch. Do a man cat for more info, because you can also get line numbers, for instance. The first file below has CRLF, which shows up in cat’s output as ^M$, and the second file has only a ^, which is equivalent to the Mac CR line ending only situation. What you need will depend upon the import program.

[root@server /path/to/files]# cat -e inputfile1.csv

Part,Cust,Price,StartDate,EndDate,Reason^M$

123-ABC-456,CUST000001,100,6-01-2010,05-31-2011,Regular Update^M$

456-ABC-789,CUST000001,100,6-01-2010,05-31-2011,Regular Update^M$

…

[root@server /path/to/files]# cat -e inputfile2.csv

Part,Cust,Price,StartDate,EndDate,Reason^123-ABC-456,CUST000001,100,01-06-2010,31-05-2011,Regular Update^

Besides line endings, there is also the text encoding of the file, to watch out for. For instance, is the file saved in Roman or Unicode or some other format? In the end, take care to confirm the file you have output is what is needed by the program for input. Enjoy!

After a spate of Mail.app problems probably related to having too damn many mail accounts and messages, I had to do the "Mail.app Reimport Samba" taking hours to let Mail.app reimport the entire mail store. After that got resolved, I found out that my Mail.app "Entire Message" selection in Search was greyed out. This turned out to be a symptom of a problem with the Spotlight index, since this search function of Mail.app is dependent upon the Spotlight index of the hard drive where your mail is stored. Generally speaking, the Solution was to delete the Spotlight index, and then re-index the drive. However, it was not a simple process (is anything?!), and as such was very "un-Apple", so I thought I would take the time to document it.

How to Make Spotlight Re-index Your Drive, Even When She Says No!

A.k.a., How to Fix Greyed Out "Entire Message" in Mail.app Search. As I was learning to do this procedure in my OS X 10.6.3 system, looking at the Apple forums and elsewhere, I observed that Spotlight was taking "forever" to re-index, while it should re-index a 500GB drive in an hour or so. When I clicked the "pulsating" Spotlight icon in the upper right of the screen (it pulsates or winks while re-indexing), I saw a message that Spotlight was calculating the time required to index the drive with a barber pole progress bar, but it was just stuck in this state for days. This prompted me to try various things to fix it, and no one method from any one forum ever worked for me, in practice.

So, let me try to explain what did work.

Remove the Extraneous

Disconnect any external drives and shut down any unneeded programs.

Remove Unneeded Spotlight Importers

Spotlight importers which allow programs to get their output files indexed are sometimes the cause of Spotlight crashes. You can find them in /Library/Spotlight and ~/Library/Spotlight (where ~ is your home directory, of course). If you have third-party importers, especially in ~/Library/Spotlight, you can move them to another folder for safekeeping, then move them back one by one to see if the "indexing forever" problem recurs. I deleted ones that were associated with programs I never use, but kept the ones I do use.

You can search the entire drive for the .mdimporter files, using this command. Hat tip to "Hal Itosis" (lol) on the MacFixit Forum.

# find -x / -iname \*.mdimporter -exec ls -lndotT {} +

Repair Disk Permissions

I use the excellent Cocktail for this but you can do it in 3rd Party OnyX or Apple's included Disk Utility as well. In Cocktail, use the Disks menu, Permissions tab. Choose to Reset Permissions and ACLs for Home Directories for All Users, then click Repair. This takes about 30 mins to execute.

Prepare the Index

Using Terminal, use launchctl to unload the indexer by controlling the appropriate launchd command, use mdutil to turn off indexing for the root folder (the /) of the local hard drive, use rm to delete the index itself (.Spotlight-V100 in the root of the drive), and finally trash Spotlight's plist.

$ sudo bash

Password: *******

# launchctl unload -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

# mdutil -i off -E /

# cd /

# rm -rf .Spotlight-V100

# rm -rf ~/Library/Preferences/com.apple.spotlight.plist

# exit

$

To explain, note the sudo bash and exit lines, bookending the procedure. This gets you a root prompt (the #) after you enter the admin password and then exits the root prompt at the end (returning to the $ prompt). You can also use sudo before every command, to be extra safe.

Clear All Caches

Noting that this part will force a reboot, again in Cocktail Files, Caches, choose Options and select all caches you can. Executing this resets the system to a state close to what you get when you install anew or do a major upgrade. Note that the system will take longer to restart because it is rebuilding caches, and applications will be sluggish the first time they restart, because they are rebuilding font caches.

Click Clear to execute the cleaning procedure and wait while it finishes. Choose to let the system restart, but when you hear the startup chime, hold down Shift so the system restarts in Safe Mode.

Safe Mode Index Rebuild

The system will take a while to get into Safe Mode. After you press and hold Shift while starting up, once you see the progress bar during startup (which is not present in normal mode), you can let go of Shift and go get coffee while it starts up. The login prompt will have a red Safe Mode in it, to alert you to the difference.

Again in Terminal, issue some commands to rebuild.

$ sudo bash

Password: *******

# launchctl load -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

# mdutil -i on -E /

/:

Indexing enabled.

# exit

$

After a few minutes you should see the Spotlight icon start to pulsate, but this time, the "Calculating Indexing Time" should change to some value, and a blue progress bar, which means the indexing is actually working. You can restart into regular mode, and the indexing just picked up where it left off, for me. However, do not connect any additional drives until it is done indexing the main drive.

Observing and Confirming

There are a few things you can do to observe and confirm Spotlight's activities, in Terminal. This assumes you are at a root prompt.

Check the size of the Spotlight index using du (directory usage):

# du -hsc /.Spot*

1.1G /.Spotlight-V100

1.1G total

Note that this should be a fairly large file. If it is only a megabyte or so, something is broken or your Spotlight index is off.

Check Spotlight-related processes with ps (process lister):

# ps axcru |sed '1p;/ md/!d'

USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND

root 352 0.0 2.8 3696728 117196 ?? Ss 8:44AM 24:25.73 mds

_spotlight 6677 0.0 0.2 2532748 10292 ?? SNs 3:19AM 0:00.33 mdworker

rcogley 6664 0.0 1.2 2638620 48240 ?? SNs 3:19AM 0:02.95 mdworker

Check index status with mdutil.

# mdutil -as

/:

Indexing enabled.

/Volumes/COGLEY-WD:

Indexing enabled.

/Volumes/COGLEY-WD/Backups.backupdb:

Indexing enabled.

/Volumes/COGLEY-WD2:

Indexing enabled.

/Volumes/COGLEY:

Indexing enabled.

List open files in /System/Library related to Spotlight, using lsof (list open files).

# lsof -c md |grep -v /System/Library |grep -v Spotlight

Get general system information with df and diskutil:

# df -lh

# diskutil info /

Show commands you have entered in the bash shell:

# history

Note that if you use an alternative shell like fish, some of the above commands will not work. From fish, just do sudo bash to get a bash-based root prompt. Be sure also to check the Console GUI application in /Applications/Utilities. This will show crash information for the mdworker program, which may give you hints to what is going on.

I hope this article helps someone, especially since this problem seems to be pretty prevalent, these days. Enjoy!

In the wake of the Zendesk Price Fiasco of 19 May 2010, in which SaaS helpdesk vendor Zendesk announced pricing hikes which negatively affected most of their users (and would have caused a doubling for us), and after which an angry firestorm erupted, Zendesk users received an email apology from CEO Mikkel Svane today, and more information in his blog post.

I have a bit of a bitter taste in my mouth after having been treated that way, but let's let bygones be bygones. I very much appreciate the real grandfathering they have extended to existing customers.

So, apology accepted, Zendesk, and kudos to you Mikkel for doing the right thing. I am sure it was difficult.

After the Zendesk Price Fiasco of 19 May 2010, where Zendesk are proposing to significantly raise prices for their SaaS helpdesk solution, I am confident many users are looking elsewhere. This is one of the pitfalls of a SaaS solution, wherein the provider achieves external investment, and subsequently bows to pressure from said investors to raise prices or otherwise change a model that was working. At least for we users.

In our case, the cost of Zendesk, while was higher than others, was justified because it somewhat supported the Japanese mails my firm gets. I say somewhat because certain mails would cause Zendesk to have a fit, but that is mostly calmed down now.

That being said, now that Zendesk has basically doubled their prices for us and while I have made no decisions on whether to accept their "offer" of prepaying annually to lock in the current pricing for one year, I thought I would list alternatives and whether they support Japanese.

It is important to note, that there are several aspects to "supporting Japanese" some of which include:

• User Interface
• Ability to Email in and out of the Helpdesk
• Ability to search
• Ability to store Japanese data

Many web based products, and not just Helpdesk products, face these challenges when localizing to multi-byte character sets. User Interface is straightforward, as is the ability to store the data. These are kind of "level 1" localizations. Searching is harder, because of the space-less nature of Japanese, and emailing in and out is a major pain. Often, they get the email body correct, and you can see Japanese in it, but, the subject line is munged. It is a complex undertaking, especially for non-native speakers. So, caveat emptor.

List of Helpdesk Software that Supports Japanese

Confirmed Positive:

Zendesk themselves. Most Japanese mail is processed correctly but there are some glitches occasionally. We have personally confirmed all the above-mentioned aspects of Zendesk.

Kayako's Savin Behal replied to me in an email: Our software supports multiple languages. You may download the required (Japanese) language packs from the Kayako Forums at “http://forums.kayako.com/f50/”. These language packs have been shared by our clients. However, we regret to inform you that we do not support these packs officially.

Active Campaign supports Japanese fully, according to Jason in the comments on this post.

Web Helpdesk says they do indeed support Japanese email. From Shiraz Hemani, Business Development Manager: "Per your questions on email, yes Web Help Desk fully supports monitoring of any number of incoming mail accounts in all language character sets, including Japanese. Each incoming account can be associated with any number of outgoing mail accounts as well. We have more info available for that feature at:

http://www.webhelpdesk.com/trouble-ticket-software/convert-email-to-ticket.html"

Atlassian Jira does support Japanese. We had been using it in Japan, but moved away from it due to the glacial pace of development of certain security-related enterprise features we needed.

Confirmed Negative:

Tender says they do not support Japanese. Will says: "... I'm sorry, but we don't currently have international language support in Tender. We do hope to add the feature in the near future, but it's not ready at this time."

Unconfirmed:

http://www.cerberusweb.com/

http://www.thevisionworld.com/

http://www.autotask.com/software/service_desk.htm

Please comment if you have more or better information.

Updates

21 May 2010 - Zendesk has since recanted, apologized, and offered real grandfathered terms for existing customers. I have to say I have a bitter taste in my mouth, but, I would be remiss if I did not acknowledge the difficulty of changing systems. We'll sit a spell.

I got an email from Zendesk in the middle of the night and this blog post, breathlessly announcing "new features!" and along with that, new pricing. I and many other users could probably have dealt with an x% increase, but a doubling? It was even worse for people on other plans, as well, and that means a lot of upset users. CEO Mikkel Svane's patronizing attitude is not helping matters either (see the screen shot of his arrogant tweet in this post).

Down to brass tacks: we have 8 agents on the Plus+ plan and are paying the equivalent of 27 USD / agent per month. The new announced pricing puts it at 59 USD / agent per month. Users are being given an "opportunity" to lock their existing pricing in, by paying annually for a 15% discount, and it appears that the limit to this is one year. So, we take the hit up front by paying up front, and we take another hit later anyway. Sounds like a plan, Zendesk.

This is a one-sided, sneaky and very much not transparent decision on pricing by Zendesk, which I suspect is being driven by chop-licking investors. Who can trust Zendesk now that they have shown how they will treat customers. If we take the offer being dangled, who knows what will happen after their next round of investment.

Unacceptable way to run any business, and a slap in the face. I hope Zendesk will respond in a positive way, instead of burying their collective head under the sand.

Updates

The lens cap that comes supplied with the Voigtländer Color-Skopar 20mm SL II does not fit, when you have the LH-20 lens hood and a filter on. What a dumb design oversight. However, if you get a Nikon LC-52 52mm lens cap, that fits right in there, no problem. Nice.

I like the design of the Nikon caps better than the Cosina Voigtländer or Carl Zeiss ones anyway. Easy to grab and avoid getting my paw prints all over the lens.

I hope this helps any owners of the tiny high-performance Voigtländer Color-Skopar 20mm SL II. Enjoy!

I just noticed that Eurobiz Japan inanely published my article on Interim IT Management Services in their May 2010 issue under the name "Rick R. Colgey". They took the time to painstakingly "correct" my American spelling during proofreading, then got my name wildly incorrect in the article. Thanks a bunch, folks.

So, if you are looking for "Rick R. Colgey" of eSolia and not finding him, that would be me. Please see the contacts below.

Rick Cogley

CEO, eSolia Inc.

http://rick.cogley.info

http://www.esolia.com

Sincerely,

If you use OS X Mail to send Japanese email, you may find that some recipients complain that your Japanese is "mojibake" or garbled. If you are getting this feedback you can set your default Character Set encoding in the terminal.

How to Set Mail.app's Default Charset

Here's how to change it. Shut down mail, then in Terminal:

$ defaults write com.apple.mail NSPreferredMailCharset "ISO-2022-JP"

That will set Mail's default character set to ISO-2022-JP instead of UTF-8, and after you restart Mail.app, you will be golden. I tested this in Mail.app 4.2 in OS X 10.6.3 and it works for me (at least according to my Outlook-saddled colleagues).

Enjoy!

I like Numbers, from Apple's competent and beautiful iWork suite, but there are still things where it is different from Microsoft Excel, and hence it feels somewhat unfamiliar at times. For example, although Excel has never been good at handling text files saved in UTF-8 format (a big deal for those of us who work in Asia), it does have a nice Wizard for importing CSV or TSV text files.

Some applications export text files with tab-separated values, and put a CSV extension on them. Actually, CSV stands for "Comma Separated Values" where these are really "TSV" or "Tab Separated Values." When you try to import such a CSV file, Numbers will mash all the fields into one cell in the left-most column. That makes sense, because it's looking for commas judging by the CSV extension. Finding none, it just lets the data pile on.

How to Import a Tab-Separated "CSV" in Numbers

If you have a tab-separated CSV, here's how to open it:

1. Save the file somewhere you can find it.
2. Rename the file in Finder, so that it has a .TXT extension.
3. Ctrl-click the file and choose "Open With" and "Numbers".

Now the data will open correctly, with each field getting its due, and you'll get the added benefit of Numbers not munging any Japanese or Chinese characters.

Hope this tip helps someone. Enjoy!

In Mac OS X, you may have noticed if you use the Terminal that OS X automatically picks up what it thinks your hostname should be and sets it. This is nice, but the problem with it is if any utility uses your hostname to set config files, you'll have a different config file every time.

How to fix a "Strange Hostname" in an OS X Prompt

If you are getting varied prompts that look like this...:

rcogley@em60-123-194-6 ~>

...where what you're expecting is something like this:

rcogley@rickmac ~>

To fix this, you can use scutil. Here's how I did it:

scutil --set HostName "rickmac.esolia.net"

Substitute the rickmac bit with your own hostname and domain and you'll be good to go with a static prompt. Try these also:

scutil --get HostName

scutil --get LocalHostName

scutil --get ComputerName

man scutil

I hope this helps someone. Enjoy!

My upgrade to Apple's Aperture 3 came over the weekend so I upgraded and started letting it analyze faces using the iPhoto-inspired "Faces and Places" feature.

Here's a few observations from the upgrade process and just a little use of the Faces feature.

• Aperture 3's icons are colorful, compared to its predecessor. I think they still look professional, but they are a little "friendlier" and more iPhoto-like.
• Aperture 3 itself is just under 1GB in size but the sample library is about 7GB.
• To use the new features of Aperture 3, you must upgrade your Aperture 2 library. This can take several hours and did for me on a 50GB library.
• Backup "Vaults" seem to also require a full refresh, probably due to the library upgrade. Vault backups still run in that irritating modal dialog box that pops up and interrupts. Best to freshen Vaults when you don't have to work on anything else, but I still love the ability to have multiple Vaults. Note, Vaults are freshened in serial - it seems to do one, then do the next.
• Both TimeMachine and Spotlight indexing get kicked into overdrive because they detect upgraded files from the Aperture 3 upgrade activities.
• With the new Faces feature, you have to train Aperture 3 to use it. If you pick a folder of photos in your library with people you commonly photograph, and spend the time to tell Aperture who is who, you can then use the "Confirm Faces" feature to drag-select vast swaths of matching faces, or, to toggle a face to be "not Jim" or "not Jane". It works better the more you train it, and it's fun to see who it "thinks" you are. Family resemblances can be telling :-). Faces also links to Address Book entries.
• Aperture 3 is most definitely snappier compared to its previous version, and registers in Activity Monitor as Intel 64-bit. My library uses about 250MB of memory.

I think it's a worthwhile upgrade just for the speed increase.

If you use Time Machine on OS X, you need to ensure your target drive is formatted with the correct Master Boot Record type. Time Machine requires either "Apple Partition Map" (works with PowerPC or Intel but is best for PowerPC) or "GUID Partition Table" (works with Intel).

There are a couple of problems which lead to this requirement biting people on the you-know-where.

• Pre-formatted drives that will work with a Mac are not often formatted with "Apple Partition Map", but will be recognized by your Mac, lulling you into a false sense of success and security.
• Time Machine rudely does not warn you that your drive has a problem. It will happily back up for a while, then fail with some not-so-useful error.
• Re-formatting a drive in the normal way using "Erase" in Disk Utility will just erase the content and not re-do the partition map.

So, when prepping a drive for Time Machine use, you need to use the "Options" button to set the Partition Map.

Formatting a Drive for Time Machine Use

Here's how to set it up.

1. In Disk Utility, select your new hard drive (the drive, not the partitions below it in the selection tree).
2. Select the Parition tab, which you can see in the accompanying graphic, and choose the number of volumes from the "Volume Scheme" pop-up menu.
3. Click "Options", then choose "GUID Partition Table" for an Intel-based Mac or "Apple Partition Map" for PowerPC- or Intel-based Macs. Then click "OK" and "Apply".

I hope this helps someone avoid the trouble I had getting Time Machine working smoothly. Enjoy!

Many upgrading Mac users have reported that Mail is "slow" in Snow Leopard 10.6. There are several things you can do to remedy the situation. Here's what you can try, but please make sure you have Time Machine backing up your system, or are running an alternative like SuperDuper! or CarbonCopyCloner.

Reset SMTP Mail

One thing that sharp users observed on Apple's forums was that newly-created Mail accounts were not experiencing the slowness to send, that upgraders were commonly experiencing. You can export your mail, recreate your accounts and re-import everything, but another way to mimic creating a new account is to re-set SMTP credentials. It's a bit voodoo, but it seems to work.

• In Mail app open Preferences, choose the Account you are having trouble with, and then choose "Edit SMTP Server List" from the "Outgoing Mail Server (SMTP)" drop down, in the Account Information panel.
• Select the SMTP server you are using, and re-enter its credentials.
• Click OK to Save.

Do this for all your SMTP servers, and remember you can always use Keychain Access to confirm saved passwords.

Vacuum That Index

Mail keeps an index of your messages in a sqlite database, and you can "vacuum" that index regularly to compact and clean up. This is especially useful if you regularly delete mail, and is well-documented on various Mac-related web sites. Quit mail, then from Terminal, run these commands.

yourhost:~ youruser$ ls -lah ~/Library/Mail/Envelope\ Index 
yourhost:~ youruser$ /usr/bin/sqlite3 ~/Library/Mail/Envelope\ Index vacuum;
yourhost:~ youruser$ ls -lah ~/Library/Mail/Envelope\ Index 

The bookend "ls" commands just show how large your Envelope Index is in megabytes, so you can see the before and after, when running the vacuum command. The middle sqlite3 command vacuums the index. For reference, recently vacuuming my mail envelope index required about 10 minutes, but reduced its size from about 70MB to about 40MB.

Run Cocktail

And finally for general performance, you should regularly run Cocktail. From Maintain's site:

Cocktail is an award winning general purpose utility for Mac OS X. It is a smooth and powerful digital toolset with a variety of practical features that simplifies the use of advanced UNIX functions and helps Mac users around the world to get the most out of their computers. Cocktail is installed at more than 200 000 computers world wide. The largest part being private individuals, but Cocktail can also be found at large international companies (Puma, Sony), educational institutions (Harvard University, University of Texas) or newspapers (The New York Times, Business Week).

The application serves up a scrumptious mix of maintenance tools and interface tweaks, all accessible via a comprehensive graphical interface. Most of Cocktail's major features are arranged in five basic categories. In addition, a Pilot lets you clean, repair and optimize your system with one click of the button.

Cocktail's Pilot is where you can schedule commands to run. I do a weekly Cocktail run, to reset disk permissions and delete caches. It seems to keep things running smooth, but you should note that for certain system cache resets, you should restart the system afterwards.

At any rate, I hope the above information helps someone out. Enjoy!

Apple's pro photo management application Aperture 2 is long overdue for an upgrade, and I hope that we'll see Aperture version 3 before long. But while I'm itching for new features and functions, Aperture 2 covers the bases for me, in spades. Aperture has a nice feature called Vaults, which lets you automatically back up your photo library with all metadata to an external drive or drives. Except when it doesn't. For whatever reason, I was having trouble saving to one of my vaults saved on an older external FireWire hard drive, and it was returning errors about not being able to create folders or write files. This sounded like a permissions problem, so I looked into it.

An Aperture vault is a "package" file in OS X, which means the file is a collection of folders and files, which appear in the Finder as a single file. The original Aperture library, the iTunes and iPhoto libraries, and files from applications like Keynote or Pages are like this. Note I'm using OS X Snow Leopard 10.6.2, but these instructions should work on Tiger or Leopard as well. Here's how I dealt with the error.

How to Refresh Permissions on your Aperture Vault

The first thing is to shut down Aperture. Since Aperture grabs the Vault file and holds it open, it might be locked. Looking at the MyVault.apvault file in Finder, I can see it's locked by doing cmd-I and digging around in the information panel.

Recursively unlock. Because you need unlocked files to perform permission settings, you can start by recursively unlocking a folder, like this. Use sudo if you are not logged or su'ed in as root (run "sudo bash" to do this but be careful), and you can unlock files in an entire volume by cd-ing to /Volumes first if need be. Of course you can do this in the Finder's information panel as well.

chflags -R nouchg /path/to/folder/MyVault.apvault

Set POSIX owner and group. Comparing to a working Aperture Vault, the basic permissions were your usual user account, and "staff" as the group. Change the owner to your account with the group as staff. The -R makes it recursive even inside the vault package file.

chown -R rcogley:staff /path/to/folder/MyVault.apvault

Set POSIX basic perms. After setting the owner and group, I set the vault's permissions to 777.

chmod -R 777 /path/to/folder/MyVault.apvault

After that, I re-opened Aperture, refreshed the Vaults, and it worked without a hitch. I hope this hint helps someone. Enjoy!

• blog:Cogley - Handle Leading Zeros in Apple Numbers

  Apple's iWork '09 Numbers spreadsheet is a versatile app with a lot of power available if you open your mind and don't expect it to be Excel. It does not quite do everything Excel does, but it handles UTF-8 well (where Excel does not and has never), and I take advantage of that often. I also love the formatting options and the multiple-sheets-per-document paradigm, but that is a different post. One challenge in both Excel and Numbers is how to handle fields with numbers with leading zeros. For instance, a part number 001234 will come out as 1234 when you import it from a CSV in either app, and lose meaning if the actual part must include the leading zeros. You can set a cell or column format in Excel as 000000, and this works the same way in Numbers, except the method's a little unfamiliar. How to Format a Part Number Field to Preserve Leading Zeros

  tags: apple, iwork, numbers, zeros, text, format

Apple's iWork '09 Numbers spreadsheet is a versatile app with a lot of power available if you open your mind and don't expect it to be Excel. It does not quite do everything Excel does, but it handles UTF-8 well (where Excel does not and has never), and I take advantage of that often. I also love the formatting options and the multiple-sheets-per-document paradigm, but that is a different post.

One challenge in both Excel and Numbers is how to handle fields with numbers with leading zeros. For instance, a part number 001234 will come out as 1234 when you import it from a CSV in either app, and lose meaning if the actual part must include the leading zeros. You can set a cell or column format in Excel as 000000, and this works the same way in Numbers, except the method's a little unfamiliar.

How to Format a Part Number Field to Preserve Leading Zeros

Here's how to not maim your part numbers.

1. Select your column to format, and open the Cells inspector.
2. Select Custom Format from Cell Format then click Show Format.
3. Give the format a name, and choose the base type.
4. Delete whatever format is in there by default and drag up an Integers type lozenge.
5. Open the disclosure triangle, and choose "Show Zeros for Unused Digits" and you will see the #,### change to 0,000. Click Show Separator to deselect it and remove the comma. Add two digits.
6. Click OK to save and apply the format to the selected column.

If you set the format as 000000 for a field that includes six digit numerics with leading zeros, and a mix of text with numeric part numbers, such as:

001234

P098765

005544

R-09-PCX

... the latter will not be affected by the format, which is just the right behavior we need.

I hope this tip helps someone, because not being able to set this really drove me a bit batty. Enjoy!

CocoaTech's Path Finder tool is a versatile Finder replacement. One problem that you might have if you do any work with data, is importing CSV files that are in the UTF-8 format, and which contain multi-byte characters such as Japanese, into Excel.

To import a UTF-8 CSV into Excel, you need to re-save into a format that Excel will accept, because it ironically does not accept the quite-universal UTF-8. I tried opening my UTF-8 CSV with TextMate and Text Edit to do the re-save into a different encoding, but neither of those allow me to save to Shift JIS, which renders Japanese characters so Excel can import them properly.

I saw that Path Finder has a native Text Editor, and thought I would try it. Sure enough, it allows you to re-save a file in Shift JIS and with a TXT extension, which can then easily be imported into Excel, unmunged.

Thanks CocoaTech!

Apple's OS X Snow Leopard Server 10.6 implements Greylisting, an anti-spam technique based on forcing sending SMTP servers to "slow down" before they can deliver. This is great for reducing spam, but it also has the perhaps undesired effect of causing delivery delays. Sometimes really, really loooong delivery delays.

In SLS, when you enable anti-spam in your Mail server (which is postfix), greylisting is automatically enabled. Because there are no readily available manuals on how to use this feature, from Apple, you may want to turn it off. Note that I'm skittish about changing config files like in a normal Unix server in an Apple server, because Apple is known to simply change vast portions of their server products without much notice. It's possible that you'd spend time implementing, and they change the way it has to be done so you have to redo it. Anyway, here's how to disable:

How to Disable Greylisting in Snow Leopard Server

Of course, as implied above, you can stop Greylisting by turning off spam filtering altogether. However, to be more specific and just disable Greylisting, do the following:

1. From Terminal on the server (ssh'ed in or direct), do "sudo bash" to login as root. Then use nano to edit /etc/postfix/main.cf
2. Remove the "check_policy_service unix:private/policy" string from the line that starts with "smtpd_recipient_restrictions" near the bottom of the file. Save, and exit nano.
3. Issue a "postfix reload" to reload the configuration.
4. Use the "exit" command to quit the sudo bash root shell.

I'm a little miffed that Apple would enable this by default and not implement any easy way to edit the greylists or whitelists. At any rate, you can read a couple articles on greylisting, or just wait for Apple. Time however, waits for no man. :-)

I use and love the text editor Textmate, which has some powerful functions. One thing that it can help with is quickly editing text files, and for example today I used it for searching lines in a mail system's "aliases" file. I wanted to remove 50-odd lines with the word owner in them, so I used the Find command with Regular Expression checked.

The search string is:

^.*owner.*$

If you enter that string which means to find the lines with owner in them, check "Regular Expression," and leave a blank in the Replace box, Textmate will blank out the lines for you. Convenient!

• blog:Cogley - Recovery of Corrupt Apple Leopard Server Open Directory

  Corrupt Apple Leopard Server Open Directory Services Thu, Oct 15 2009 22:24 | LDAP, Open Directory, tips, software, Troubleshooting, apple | Permalink I had a Leopard Server crash and burn so that nothing was responding, and when I forced the server to reboot (as well as rebooting a bunch of other ancillary servers and services just in case), I found an ominous sign in Server Admin, along with no user accounts in Workgroup Manager. Eek! Server Admin's Open Directory showed:

  tags: open, directory, apple, leopard, server, recover, crash

I had a Leopard Server crash and burn so that nothing was responding, and when I forced the server to reboot (as well as rebooting a bunch of other ancillary servers and services just in case), I found an ominous sign in Server Admin, along with no user accounts in Workgroup Manager. Eek! Server Admin's Open Directory showed:

LDAP Server is: stopped

Password Server is: running

Kerberos is: stopped

Not good. Never fear, though.

How to Fix a Corrupted Open Directory

First, don't panic. Apple's forums show you can use "

slapd -Tt

myhost:~ administrator$ sudo bash

Password: ********

bash-3.2# /usr/libexec/slapd -Tt

overlay_config(): warning, overlay "dynid" already in list

overlay_config(): warning, overlay "dynid" already in list

overlay_config(): warning, overlay "dynid" already in list

overlay_config(): warning, overlay "dynid" already in list

overlay_config(): warning, overlay "dynid" already in list

bdb(dc=myhost,dc=mydomain,dc=com): PANIC: fatal region error detected; run recovery

bdb_db_open: Database cannot be opened, err -30978. Restore from backup!

bdb(dc=myhost,dc=mydomain,dc=com): DB_ENV->lock_id_free interface requires /

an environment configured for the locking subsystem

backend_startup_one: bi_db_open failed! (-30978)

slap_startup failed (test would succeed using the -u switch)

The "run recovery" here means to run the

db_recover

slapd_db_recover

bash-3.2# db_recover-v -h /var/db/openldap/openldap-

openldap-data/ openldap-slurp/

bash-3.2# db_recover -v -h /var/db/openldap/openldap-data/

db_recover: Finding last valid log LSN: file: 6 offset 4190936

db_recover: Recovery starting from [6][4190795]

db_recover: Recovery complete at Thu Oct 15 21:57:41 2009

db_recover: Maximum transaction ID 80000225 Recovery checkpoint [6][4190936]

Ah, that looked nice. Then run

slapd -Tt

bash-3.2# /usr/libexec/slapd -Tt

overlay_config(): warning, overlay "dynid" already in list

overlay_config(): warning, overlay "dynid" already in list

overlay_config(): warning, overlay "dynid" already in list

overlay_config(): warning, overlay "dynid" already in list

overlay_config(): warning, overlay "dynid" already in list

config file testing succeeded

bash-3.2# exit

myhost:~ administrator$

After a few minutes

launchd

LDAP Server is: running

Password Server is: running

Kerberos is: running

A couple of tests shows I once again have Wiki Server, iCal Server, Jabber Chat etc, all the Open Directory and Kerberos-based services back on line. Breathe a sigh of relief if this helped you and let me know in the comments!

• blog:Cogley - Linking File Types and Apps in OS X

  How to Re-associate File Types with Applications in OS X - If you are an OS X user, and you find files of a certain type, say PDFs, are opening in one applications but you want them to open in a different one, you can easily change the association using Finder.

  tags: os, x, apps, association, file, type, tiger, leopard, snow

If you are an OS X user, and you find files of a certain type, say PDFs, are opening in one applications but you want them to open in a different one, you can easily change the association using Finder.

How to Re-associate File Types with Applications in OS X

Here's how:

1. Select a file in Finder and ctrl-click it.
2. Select "get info" from the context menu.
3. Find the "open with" section in the "get info" menu that appears.
4. Select your desired application from the drop down list.
5. Click "change all" to set the association between that file type and the application you selected.

This has worked in OS X Tiger, Leopard and Snow Leopard. You can use this method to, say, open all PDFs in the native OS X "Preview", Adobe's Acrobat, or Skim, for instance. Please leave a comment if this helped you. Enjoy!

Of course it should not be used for nefarious purposes, but Apple's Safari browser makes it trivial to download all the images or files on a web page you are visiting. I had The Logo Factory create a special logo for my company eSolia's 10th anniversary, and they prepared a page with the deliverables on it. I did not want to download each one individually, and I remembered that the Safari Activities window allows you to access the objects on a page directly, such as various file attachments on a page.

O' Sensei of Safari, How Do We Achieve this Magic?

You can use Safari's Activity and Downloads windows, both available in the Window menu in Safari, in this way:

1. Browse the page you want to download from, then open Activity from the Window menu.
2. Find the page among the other pages if you have multiple tabs open. Use the disclosure triangle to open the outline of the objects on the page.
3. Find and select your download targets. Press Cmd-C to copy to clipboard.
4. Display the Downloads dialog, also available in the Window menu, then paste into it. Cmd-v.

You should see the images or files start to download in the Downloads window. I hope this is helpful to someone.

Is Directory Utility, which has been available in /Applications/Utilities, missing in action in Snow Leopard? No, it's just been moved to Core Services. Access it this way:

1. Open Apple Menu, System Preferences.
2. Enter Accounts, clicking the lock to authenticate as needed.
3. Click Login Options at the bottom of the accounts list.
4. Click Edit, to the right of "Network Account Server."
5. Click Open Directory Utility.

You use Directory Utility to connect to Active Directory, Open Directory, or other LDAP servers.

• blog:Cogley - Changing a Bike Inner-tube

  I got a flat the last hill of my 100 km bike trip last Sunday. Thank heavens it did not happen at km 50 or something. I went to a bike shop in Shinjuku today to get a replacement tube, and they were kind enough to tutor me on how to replace it. How to Change that Tube Here's the process I learned at the bike shop: Purchase a tube, tire levers (they come in sets of three, usually) and rim tape of the appropriate size. My rims are 26 inch with 1.5 Schwalbe Marathons on them now, and you just have to be sure what you buy is the right size. If you can give them the rim size, that's better too. The tubes come with various valves, and I have "French" valves now so that is what I got. All told, the cost to buy the parts was about JPY 1300 (USD 13).

  tags: bike, cycling, inner-tube, change, rim, tape, spoke

• blog:Cogley - Fixing EMobile USB Dialup on Snow Leopard

  I just installed Snow Leopard OS X 10.6 with no problems after getting a replacement for a bad Family Pack install disk (the Shibuya Apple Store said that many people reported the same), and found that my EMobile Huawei D02HW USB Wireless Dialup card, which was fine in Leopard, died when Snow Leopard was installed. Reinstalling the EMobile Huawei D02HW on Snow Leopard Here's how I fixed it:...

  tags: emobile, d02hw, apple, snow, leopard, huawei

I got a flat the last hill of my 100 km bike trip last Sunday. Thank heavens it did not happen at km 50 or something. I went to a bike shop in Shinjuku today to get a replacement tube, and they were kind enough to tutor me on how to replace it.

How to Change that Tube

Here's the process I learned at the bike shop:

1. Purchase a tube, tire levers (they come in sets of three, usually) and rim tape of the appropriate size. My rims are 26 inch with 1.5 Schwalbe Marathons on them now, and you just have to be sure what you buy is the right size. If you can give them the rim size, that's better too. The tubes come with various valves, and I have "French" valves now so that is what I got. All told, the cost to buy the parts was about JPY 1300 (USD 13).
2. Remove the wheel with the flat from the bike. I have Shimano Deore XT rim brakes, and there's a place you hook the brake wire's flange in, which if released, gives you the leeway to get the tire off.
3. If you have French valves, completely remove the bolt that keeps the valve in place in the rim.
4. On the opposite side of the wheel from the valve, insert a tire lever between the rim and the tire, and use it to lever the tire out, in that area. Take care not to pinch the tube while you do it, just in case you want to fix and reuse it. You'll notice there's a kind of hook on the one end of the lever - this goes onto a spoke to keep the lever in place, holding the tire edge out and away from the rim.
5. A couple of spokes away, do the same thing again with the second lever, to get more of the tire out.
6. Now you should be able to slide the third lever under the edge of the tire, and rotate it along the rim and tire edge to get the tire out. You can keep the one edge of the tire in the rim.
7. Slide the flatted inner tube out, taking care not to damage it if you want to repair it.
8. Inspect the tire inside and out for damage. There could be something sharp embedded in the tire. Remove any sharp objects puncturing the tire. In my case there were two pieces of a broken spoke embedded in the tire and in the rim tape. I could only find the one embedded in the tire by running my hand along the inside. The rim tape problem was quite obvious!
9. If you either remove the tire completely or just push it to one side, you should be able to see the rim tape, which prevents the inflated tube from working its way into the nipple bolts for the spokes. Rim tape prevents flats, but, in time it gets worn out too. If it has been mashed into the nipple bolts too much, and there are sharp edges, replace it. Rim tape is basically like a big rubber band with a hole for the valve. You can use a flat blade driver or an awl to work old rim tape out, and to lever new rim tape on. In my case, the yellow rim tape was two years old and starting to get dry, and, it had been punctured by the old spoke bit, so I replaced it.
10. Put a little air into your new tube, because it is easier to work with the tube if it is slightly inflated.
11. Insert the valve through the rim tape and rim, and put the valve bolt on to secure it.
12. To put the tire back, this time start on the valve side (removal starts opposite the valve). You can use the tire levers to get started putting the tire back into the rim, but be careful not to pinch the new tube. Having the tube slightly inflated will make things a little easier to maneuver. Once you get the tire in a little, use your hands to kind of "knead" the tire back in, working around it. Schwalbe Marathons are a little tough, as they have Kevlar inside and are consequently a bit harder rubber.
13. Check that the valve is 90 degrees to the rim. If it is angled, work the tire and rim until you can rotate it so it is perpendicular to the rim.
14. Inflate the tube to the correct psi pressure. Confirm that it's holding air and that you have not damaged the tube.
15. Deflate the tube once, then re-inflate. The bike shop said this last step really helps to prevent flats.
16. Go ride!

Hope this procedure helps someone with their tube troubles. Happy riding!

I just installed Snow Leopard OS X 10.6 with no problems after getting a replacement for a bad Family Pack install disk (the Shibuya Apple Store said that many people reported the same), and found that my EMobile Huawei D02HW USB Wireless Dialup card, which was fine in Leopard, died when Snow Leopard was installed.

Reinstalling the EMobile Huawei D02HW on Snow Leopard

Here's how I fixed it:

1. Deleted /Applications/EMobile D02HW Utility.app.
2. Deleted /System/Library/Extensions/HuaweiDataCardDriver.kext
3. Deleted Huawei folders and files in /System/Library/Modem Scripts and in /Library/Modem Scripts
4. Emptied the Finder trash.
5. Rebooted the system.
6. On plugging in the USB Modem, the system mounts it in Finder as a USB memory. Ran the installer EMobile D02HW Utility.app and got an error regarding AutoOpen. Bypassed this by opening the installer package via "Show Package Contents" in Finder, and ran the Contents/Resources/EMOBILE_D02HW_Drv_App.pkg, which is the actual installer. Now it runs with no errors. AutoOpen be damned.
7. After the install, rebooted again.
8. After the reboot, I can add the Huawei Mobile modem in Network Preferences. Phone number for these devices is "*99***1#", user name em, password em.

I read a report that you can simply change tone to pulse dialing in the existing Huawei Mobile settings (from Leopard, for instance), so maybe the failure just has something to do with a plist not working somewhere and changing that setting refreshes it, but removing and reinstalling works fine too.

Give it a try if you have trouble, and I hope this short tip is helpful for someone.

• Snap!Japan - Japan Rail is becoming more Gaijin

  I noticed something interesting. The JR East Japan announcements about the next station are done in a female voice, and she used to say the station names with proper Japanese pronunciation. The next station is, SHIMbashi. They've re-recorded some of the announcements though, seemingly with the same "voice talent", and there's a subtle difference. She now says the station names with a "gaijin" accent. The next station is, shimBOSSshi. What's up with that? Were people not getting the names right? Did some consultant trying to justify their existence tell JR that they needed to say it more like "gaijin" say it? I'd say that would be gaijin of the American English speaking variety, though. How curious. I noticed it the other day, and today it was the original way, so I am not sure what the pattern is yet. Maybe different lines have different patterns. Japanese are pretty obsessed with regional language differences, though. There's a comedy duo called "Yuji Koji" who hysterically make fun of the difference between the regions and Tokyo. Even my car Navi has a setting to make it talk with an Osaka accent. 300m saki, hidari yade.

  tags: japan, rail, pronunciation, announcement

• blog:Cogley - Hot Brass, Percussion and Visual - BLAST!

  Blast! was born from the Star of Indiana Drum and Bugle Corps, which exited the DCI circuit to form Blast!, a kind of indoor, theatre-based "Brass Theatre" troupe taking the high skills of the best drum corps performers, and performing a kind of greatest hits of drum corps, to thrill audiences everywhere. (Not to mention winning Tony and Emmy awards as well.) The Japan Blast! tour features snare drummer Naoki Ishikawa, who was a champion "individuals" competitive snare player when he marched in DCI, and who is now a featured performer in the Japan Blast! show. He's got incredible chops, and they feature him well during the Battery Battle portion of the show. The video is the percussionists performing during the break between sets, on kitchen stools and a garbage pail. Humorous. :-) The Blast! performers did all the hot drum corps favorites like "Everybody Loves the Blues", "Appalachian Spring", "Medea", and "Malaguena" as well as a number of great numbers that were new to me. Overall, the show was about 2 hours of exciting music and visual performance, which had the audience on their feet by the end.

  tags: blast!, ishikawa, naoki, star of indiana, tokyo