Microsoft's mesh-ianic complex

Whatever doesn't kill us makes us stronger. Both Nietzsche and Kanye West famously used these words.

That was the message that came across loud and clear during Microsoft Chief Software Architect Ray Ozzie's talk on Wednesday at the Sanford Bernstein Strategic Decisions Conference. Ozzie argued that competition from Google and free and open source software (FOSS) have forced Redmond to innovate instead of resting on their Windows/Office laurels. A cynic might reply that Microsoft has little choice in the matter if it wants to stay in business.

But what I found most interesting about Ozzie's talk was what he said about the future of computing. In a word: mesh. As May Jo Foley blogged at ZDNet.com, Ozzie continued to talk up distributed, mesh-like operating systems, possibly beyond Windows. To make that happen, Ozzie stressed the need for software development kits (SDKs) that can be used across many different devices. Just coincidentally, Microsoft is moving in this direction with its much-discussed Live Mesh and accompanying Live Mesh SDK.

Software developers are taking a wait-and-see attitude regarding Live Mesh, but remain hopeful in part because of the success of Visual Studio. As one of them wryly commented to BBC News technology reporter Maggie Shiels, “The proof is in the pudding but at the moment it's all demo-ware and advertising.”

Microsoft has a huge opportunity to earn some serious street-level support in the years to come by meshing well with the likes of Apple, Google and YouTube. An open mesh sandbox – supporting open standards and FOSS – will go a long way to deliver the hearts and minds of developers.

Black Duck Software Acquires Koders, Inc.

Black Duck Software has acquired the assets of Koders, Inc., the company that launched koders.com, a popular on-line search engine for open source software and other Web-downloadable code. Koders.com gets over 30,000 developers each day searching and accessing open source code, methods, examples, algorithms, and solutions in over 766 million lines of code written in over 30 languages and identified with 28 software licenses.

With this acquisition, Black Duck is doing even more to help software development teams find, reuse, and manage open source software by incorporating Koders’ search capability into our highly successful open source product portfolio. Specifically, the combination of Koders software code search engine with Black Duck’s Code Center, released in the first quarter of this year, is very exciting, and protexIP customers will benefit as well.

Black Duck will add code and metadata from the Black Duck KnowledgeBase to Koders’ search database, the industry’s most complete database of open source and third-party code, containing more than 520 million code files, representing many billions of lines of code.

Black Duck now offers the industry’s most comprehensive array of capabilities for incorporating open source software into application development:

• Code search: With Koders, Black Duck gains a powerful code search engine that can search for specific code functions or solutions in repositories across the Internet.

• Component search: Black Duck Code Center enables development teams to search a KnowledgeBase containing hundreds of thousands of open source components. Developers can internally publish a catalog of approved open source components to facilitate reuse within their own organizations.

• Fragment/File search: Black Duck protexIP automates the review of code and finds unapproved code fragments, files, or entire components that were integrated into a code base without adhering to a company’s open source review policies. This capability can be used to uncover licensing violations, security issues, unsupported open source code, and outdated code.

Going forward, we are committed to maintaining and improving the koders.com site as a free resource. All of us at Black Duck are excited to be doing even more to advance the cause of open source software. The revolution continues.

Questioning the Gartner Report

Gartner’s SaaS-versus-OSS TCO finding triggered some of the most interesting responses to a lecture I delivered yesterday at Carleton University in Ottawa, Canada, on the state of Open Source Software. (Ottawa is the home of the Senators NHL hockey team, Renegades CFL football team, Lynx MLB baseball farm team, 67’s OHL hockey team, and the Wizards soccer club – if you didn’t know.)

The students, entrepreneurs, prof's and VC in the audience asked me great questions about the Gartner report:

• Does this mean that enterprises in 2012 simply want applications residing in front of the firewall as opposed to installed on servers or storage systems in the data center?
• Does this mean that virtualization vendors’ central value proposition will fade by 2012?
• Does this hearken the return of the mainframe with its centralization and value per/MIP (old) and green (new) value proposition?
• Does this mean cloud computing is a shoo-in?
• Won’t direct OSS costs diminish as the market matures?
• Or, does this mean that OSVs are expected to hike their prices in 2010 and 2011 and shift the cost curve to make SaaS more attractive?

In truth, it is almost always a combination of these factors that result in cost curve and associated adoption curve shifts. It is my experience that SaaS applications (SalesForce in particular, but also Constant Contact, Adesso Systems, and others) require customization, additional support beyond what is provisioned, and training is almost always needed. SaaS has many direct and hidden cost categories that are similar to OSS, but the IT value proposition is significantly clearer and more favorable. This value proposition is driving the adoption of cloud computing applications such as Google Apps.

OSS consists of an operating system, infrastructure, middleware, applications, utilities, and so on. Many of these are free as in beer or free as in freedom. But OSS has hidden costs, such as training, documentation, consulting, and other costs. Did the Gartner research take these into account?

One question asked by a bright Carleton student is especially interesting and worth putting out there to readers: What about open source SaaS?

Total Growth of Open Source Software

Amit Deshpande and Dirk Riehle, OSS researchers at SAP-Labs in Palo Alto, CA, have quantitatively analyzed the growth of more than 5,000 active and popular open source software projects and shows that the total project size (measured in source lines of code), the number of new open source projects, and the total number of open source projects are growing at an exponential rate. Previous research showed linear and quadratic growth in lines of source code of individual open source projects. This work shows that OSS is expanding at an exponential rate into new application areas.

You can find this groundbreaking research here.

Maureen: Get Another Job

Recently, a reporter wrote an article about Black Duck Code Center that was fiction and written with antagonism and not journalism. It is trash or, more accurately, fiction. While she has a history of misreporting BDS and other companies’ news, this story came about without an interview, touching base with our PR firm Schwartz Communications, or even reading our press release, so I feel compelled to write this blog post.

One of the great pleasures of my job is doing interviews with the press and analysts, and subsequently reading the results. I see an interview as a collaborative project between a journalist and me, with a goal of informing readers about something newsworthy -- a new product, business relationship, market trend, or something innovative and unique. The two parties engage in this project assuming that both parties will be fair in their use of facts and opinions. No matter what the news being reported, the objective of both parties is knowledge.

There is one immutable truth: In order for journalism to work, both companies and journalists have to be objective and use facts. That requires journalists to divulge any financial relationships, prejudices, or other influences that would shape a story or blog posting. It's just the right thing to do. This is not only a canon of journalism, but the general ethical standard underlies most communications between people.

In my mind the journalists I interact with most frequently carry with them a capital “J” in their occupational title because of the deep respect I have for them. They come from an ethical and professional place. They are friendly but trained not become too close to those on their respective beats. This group is not small. It includes Scott Kirsner, Innovation Economy columnist for the Boston Globe, Darryl Taft, and Steven Vaughan-Nichols at eWeek, Sean Kerner at internetnews.com, John Waters at ADT, Don Marti at LinuxWorld, Mary Jo Foley and Dana Blankenhorn at ZDNet, Paul Krill of InfoWorld, and many others. They use facts and write articles that are accurate and informative. They understand the nature of relationship between companies and journalists.

Alas, there are some exceptions to this rule.

The title of the article that touched off this post is an article by Maureen O'Gara entitled “Black Duck's Code Center Close to Hatching: Under a pressure from HP Black Duck says it will roll out a thing called Code Center.”

O’Gara (or an editor who wrote this headline based on her article) pulled that title out of the air or someplace else. Two weeks ago HP’s FOSSology was announced, and last week Code Center introduced. This headline, and the article, is like trying to compare apples and zebras. Maybe in O’Gara's mind these events are related, but on planet earth they are not. I truly hope this is fiction and not an attack on Black Duck by O’Gara or something inspired by our lightweight competition.

O’Gara writes: “Under a bit a pressure now that HP has open sourced its own IP identification system as FOSSology, Black Duck says it will roll out a thing called Code Center by the end of the quarter.”

Let's get the chronology straight: Code Center has been under development for 22 months. The original spec was written in the fall of 2004. HP influenced nothing -- the inspiration came from our development team and pressure came from customers – they want this product from us to compliment protexIP! Time to market is everything these days.

Roll out “a thing” called Code Center? Why didn't she call our PR agency or Black Duck to get the facts on Code Center?

O’Gara continues: “Previously Black Duck has been called in as something of an after-thought.”

In one sentence she has insulted hundreds of companies that are using Black Duck’s protexIP in production, dedicating many hours to planning its deployment, integration into processes, and producing reports that were only dreamt of before 2004 when we started shipping the product.

Finally, O’Gara writes: “…Code Center is also supposed to work with whatever component usage policies a company might have and make them less labor-intensive.”

Readers might remember O’Gara’s 2005 run-in with Groklaw blogger Pamela Jones, which led more than a few to question her ethics. (Summaries of the feud are here, here and here.) Others credited her for her “dogged” questioning. In fact, O’Gara’s bio congratulates her for asking tough questions and “getting to the heart of stories.”

Personally, I’d welcome incisive questions about Black Duck and would appreciate any honest attempt to get the facts straight. If such an effort is too much, perhaps she he ought to consider another profession. I simply wish she’d stop pretending to be something she’s not -- a journalist.

Podcast Interview

LinuxWorld Podcast featuring my discussion with Don Marti about Black Duck Code Center.

It can be accessed at http://www.linuxworld.com/podcasts/linux/ and is featured at the top of the page. (12:29)

A new pair of genes

The completion of the Human Genome Project (HGP) has helped scientists to understand that all humans on earth today can be traced back to a small population of hominids living in Africa 60,000 years ago. Now we know our code – thanks to DNA.

But what about open source software? Our research at Black Duck makes it clear that if you reuse or share software, there's a better than average chance are that a few (or more than a few) open source “genes” have become part of your company's electronic DNA [link to the finished paper]:

Over 80 percent of IT organizations reuse software components[1], and if yours is like most organizations, chances are good that you have at least some open source within your code. In a recent survey of software developers[2], 77 percent said they used open source libraries, 60 percent used snippets of open source project code, and 50 percent used Linux.

But sorting through your code's “genetics” is a complicated business that requires individual lines of code to be checked against a comprehensive database to trace its origins.

Black Duck's ever-evolving open source software code database – which we leverage in all our products -- is what you might call an Open Source Genome Project (OSGP). Just as the HGP can help you trace your origins back to the grasslands of Africa and beyond (it turns out we share DNA with sea urchins and worms), the OSGP can help you trace the origins of the open source code you're using. This is useful for a variety of reasons, not the least of which are licensing requirements and software audits.

As it is with humans, so it is with software -- it's always a good idea to know your code.

---

[1] Source: Gibson Marketing Group, June 2007

[2] Source: Black Duck Software, 2006

Home Grown is Not the Best Way to Avoid Rude Surprises

Home grown might work for vegetables, but it doesn't work for code compliance. The case of Google illustrates my point.

Darryl Taft at eWeek interviewed Chris DiBona, the open source programs manager at Google, and posted an informative story and Q&A. DiBona notes he is not currently using Black Duck's products because he has implemented "tight" controls over Google's use of open source components in software development.

I often talk to companies in similar situations as Google's. Black Duck's biggest competition is manual checks -- visual code reviews -- on software development that try to ensure that code is assembled properly. Almost without exception, and even though the internal processes can be very effective, these companies select Black Duck's product after talking to us. Here's why:

• Manually driven processes don't scale. More and more companies are incorporating Black Duck's solutions to make software governance a standard part of the software development process. In this way, it's automated, and it's easy for anyone involved in the development process, including managers and other executives, to receive a report on what governance issues need to be addressed.
• Large enterprises in particular are creating their own open source software stacks, and they are implementing them in a standard way across development. I would imagine that the Google developers would love this type of standard (if they have not implemented one already) as a way to streamline their process, and it also assists with those "tight" controls on the use of open source components. A Black Duck solution can be used to ensure these standard software stacks -- including open source, third-party, and proprietary code -- are used correctly within the development cycle.
• Manual code reviews are not nearly perfect. While software developers feel "closer to the code" as a result of these code reviews, even companies with the best processes and intentions can violate policies and license obligations. We regularly talk to prospective customers that swear they have a foolproof compliance process, only to find "rude surprises" when they do a Black Duck code analysis.
• Today software development is complex -- more assembly oriented -- and getting more complex. Code made in distributed locations, third-party code, open source code, scripting solutions, and other code is all assembled these days into a great soup, and this is bound to have "rude surprises”.

There are even more reasons why Google and other companies should adapt enterprise-class automated code analysis solutions. But I will write about other examples of Google-like companies in future blog entries.

In the meantime, just remember that home grown code is not always a garden of delights – vegetable or otherwise.

Frankencode Looming

Your mileage may vary.

Don't use this on a production machine.

Cross your fingers before installing.

If you've been around the Internet long enough, you'll see phrases like these used by software developers. And we all know what these warnings mean: This is not finished yet ... Don't say we didn't warn you if your computer melts down or the software disappoints.

But what should we expect when software or a Web-based application doesn't come with such a warning label? Shouldn't users – whether at home or in the business world – expect it to deliver what it promises?

The Urban Dictionary’s definition of Frankencode is “A program or subprogram consisting of bits and pieces of other, possibly unrelated, code that only barely performs its function and is prone to errors. Much like Frankenstein's monster, such code can create havoc in a user's life.”

Sadly, it's not just newbies who give life to Frankencode. Software developers do too. (You know who you are.)

Here, we're talking about legitimate applications or well-intentioned pieces of code pushed out onto the Web without proper and thorough testing. As the philosopher Daniel Dennett might say, frankencode is a piece of code with performance anxiety.

Frankencode is showing up everywhere with greater frequency. Frankencode as a creation of our era of computing arising from, in part, the abundance of Web 2.0 widgets, web services, and code in Web-based software repositories. It has become all too easy to bolt together a few widgets or pieces of code, give the result a cool, memorable name, and set it free onto the world.

Several years ago, Wired editor-in-chief Chris Anderson coined the term the “Long Tail” to describe the niche-ification caused by the web. He writes about it on his blog:

The theory of the Long Tail is that our culture and economy is increasingly shifting away from a focus on a relatively small number of "hits" (mainstream products and markets) at the head of the demand curve and toward a huge number of niches in the tail. As the costs of production and distribution fall, especially online, there is now less need to lump products and consumers into one-size-fits-all containers.

So the long tail can be a good thing in that it gives consumers more choices. But the long tail also grows when Frankencode dupes unsuspecting businesses and consumers into thinking it's the real deal. The time and money wasted on these bogus solutions is a reminder that old-fashioned technical due diligence is the order of the day for enterprises and consumers alike.

Open Source License Compliance

Article in Line56.com
Tuesday, January 30, 2007

Open Source License Compliance

With developers freely downloading potentially hundreds of open source components per week, with or without their employer's permission, an automated software compliance management solution is necessary to establish and manage license compliance policies across the enterprise.

Bob Zurek's Technology Observations

I met Bob Zurek when he was VP of Product Management and Advanced Technologies at Ascential Software prior to the acquisition of Ascential by IBM. At IBM he’s the Director of Advanced Technologies with IBM Information Integration Solutions. He just posted one of my articles on his IBM site.