Wow, what a cool project to work on: working out the logic flows in assembler and breaking them out so that they can be translated into a higher-level language.
I've always thought that this type of government work should be open to bids from anyone. If the government is worried about trojans, malware, etc., then they can easily hire an auditor to audit the code and vouch for its authenticity. The fact that only very large, well-connected corporations get a crack at these types of problems is insane and a complete waste of taxpayer money.
The big trick is the proposal itself. Writing that is only possible if you are already tied in at all kinds of levels. I've seen some of these tenders up close, the companies that land the deals submit phone book sized proposals to tenders that are officially open but actually closed unless you are in a very select circle already. It's not uncommon for the proposal writer to then pass on the actual work to a whole slew of subcontractors at substantially lower rates.
Thanks, I suspected that this was the case after I read the Cringely book on IBM (https://www.amazon.com/Decline-Fall-IBM-American-Icon-ebook/...). It gives one the distinct impression that connections matter a lot, and that getting the contract matters more than actually completing it successfully.
The career path at my former defense contracting gig went from software engineer to someone who writes proposals. They clearly valued winning work over executing it. Rent seeking economics at work.
Almost no one wins submitting proposals out of the blue, certainly the case with unsolicited proposals but also with responses to formal RFPs. Firing in a proposal with no agency contact, no familiarity ("customer intimacy"), no shaping, no premarketing, no office calls, no digging to understand their real pain points is even worse than a cold call because, as you noted, it is horrendously more expensive and painful to prepare a large proposal.
Small contracts are an entirely different matter. Try bagging something >> $1M or defense related.
I've worked for the government on small jobs here in the EU a couple of times, as long as you stay below a certain amount you can bypass a ton of requirements. But once you go above that threshold the number of players drops very rapidly and there is no escaping the formal process.
If you are capable of selling large contracts to the US Federal Government without having to submit a typical bid proposal then you are in a very fortunate position, but the people that I know that have done this in the past have all worked on stuff that either required their fairly unique skills or they were working on extremely small contracts (< $250K).
Here is one of the two young entrepreneurs (19 and 23 years old) that secured ~$300M DoD contract. So obviously, it is not as bad. ;)
In 2005, Packouz (23 years old at the time) joined Efraim Diveroli (19 years old at the time) in Diveroli's arms company AEY Inc. By the end of 2006, the company had won 149 contracts worth around $10.5 million.[1] In early 2007, AEY secured a nearly $300 million U.S. government contract to supply the Afghan Army with 100 million rounds of AK-47 ammunition, millions of rounds for SVD Dragunov sniper rifles, aviation rockets and other munitions
Pity they had to do it fraudulently. This is not exactly a bespoke product though. 4 years in jail/7 months house arrest is not a great thing to have on your record either but I guess the payday made it worth it for them.
The intestinal fortitude needed to prepare a proposal for these mega projects is enormous. I always end up with a gut churning feeling that no one is even going to read my work (which is interrupting any work on real projects), or that some back hander will result in a not level playing field.
Do people read them? If it's really a phone-sized book, I assume that there's some sort of algorithm, whether run by a person or a computer, that is just looking for things to check off a list.
Sometimes from debriefs, it is clear that they at best skimmed. Therefore the job of the proposal writer is to make the evaluator's job easy and to make the reader eager to turn the page, e.g., no walls of text just like on reddit or Hacker News.
Are you saying this is how contracts are awarded on FedBiz Ops?
I do grant writing for nonprofit and public agencies, along with some research-based businesses, and people interested in getting into this sort of thing will contact us, or people like us.
We're a little like lawyers: it is possible to do everything right, but it's very hard and unlikely if it's your first rodeo.
I just registered on SAM and submitted a bid (something unrelated to this), but no idea what I was doing, I mean I put together a plan, had meaningful past performance, but overall I'm just hoping for the best. Would you happen to have any resources you followed? Or, suggestions that you believe led you to submitting a successful bid?
There are Procurement Technical Assistance centers in every region of the country. They are staffed by former procurement officers, and help companies learn how to sell to government and prime contractors.
I was just looking into it a few hours ago, do you have a good experience with them? Just as many people on here I've been through different forms of office hours, one of them with a locally run entrepreneurship center, I saw an attorney (that was years ago before I could properly afford one). I know the guy volunteered his time, but basically what he said was "hey, give me a portion of your company, and I'll provide you with services", I came away with absolutely zero advice. That's why when I was looking into it today, I was a bit skeptical.
I saw a presentation from one of the people from the local office. Her advice seemed reasonable: Go to the procurement conferences, get a one-pager of the capabilities of your firm, and talk to the purchasing agents. See government contracting as a supplement to your business revenue, not the core.
Custom database product for EPA, seemed like a good fit. While I have no connections to EPA, I do some collaborative work with bio-engineering department at a university in Chicago, I'm a data scientist/developer/have a small team, I thought I'd try dipping my toes in something new.
Any technical people you were able to have discussions with in preparing your bid would be good places to start with follow-on conversations.
An acquaintance with a setup similar to yours does well with a virtuous cycle of rolling SBIR and STTR results into his commercial products, which fuel more SBIR wins. He also does lots of legwork in the form of hand delivering white papers he’s written in office calls with customers and potential customers during site visits.
People do business with people — particularly those we like, know, and trust — not companies and agencies. Find someone whose headache you can make go away. Keep the conversation moving.
This is a patient person’s game. Sometimes you’re planting seeds that will bloom later.
This seems like a problem of trust and the IRS has already chosen whom it trusts based on past work done etc. Seems unfair, but how would you be able to trust someone whom you’ve never done business with earlier? Could these be translated to laws to prevent this type of exclusion?
> If the government is worried about trojans, malware, etc., then they can easily hire an auditor to audit the code and vouch for its authenticity.
Not that I disagree with the general idea of democratizing this sort of big government contract, but, well, this is a much bigger hurdle than you are making it sound like. Audits can be good at finding unintentional flaws, but a skilled adversary can often create code that looks safe, but in fact, isn't. Consider the underhanded C contest: http://www.underhanded-c.org/
On a personal note: I once worked on a large research project for detecting malicious behavior on Android apps, where the idea was to produce tooling to find underhanded/undocumented behavior automatically. The project had a red team. By the final rounds, we were getting code from them where the malicious functionality was extremely hard to find via either tooling or manual inspection. Keep in mind these were simple programs, rarely over 10k lines of code, written in Java which is a remarkably transparent language to read, and that we were allowed to ban features like reflection or raise the alarm on anything that looked like intentionally obfuscated code. Additionally, we knew each of those programs had malware in them. Reporting 'clean' was often just saying 'you win, we give up, couldn't find it after staring at this 2k lines file for a week'.
In theory, when working with a large contractor, you can put controls on how the software is built, not just the final code, and you can hold them accountable for backdoors discovered long after the fact. Now, not saying this always works that way, but it might still be better than accepting a system built by someone you only know from their Github handle and who might or might not live within your jurisdiction. The state of the art in software verification would need to change a great deal before that is a good idea.
My thinking is that the issue isn't actually technical, but one of risk management, and that's something that markets are very good at dealing with. I would guess that there would be quite a few companies out there willing to step in and certify/insure code quality, for the right price.
Of course, DOD and other more sensitive systems are a completely different animal with respect to risk, so this wouldn't necessary apply there.
I would disagree. On of my first jobs was porting assembler to C for a paging switch. The assembler was well written and well documented. The task was slow, hard and painfully boring. It took months to get every function to be a perfect match.
Manually, sure, but it sounds like the idea is/was to use an automated approach. I imagine many of the same techniques used by modern decompilers could be applied, and there's probably lots of information in assembly that is lost in machine code (e.g., JE vs JZ on x86). One could also assume some knowledge of the coding conventions used by the organization. Even a tool that could translate 90% automatically, leaving the rest to be done manually, would work out pretty well.
In some ways it reminds me of the project to programatically translate the golang compiler from C to Go. Though I imagine the codebase is quite a bit messier!
Did you manually convert the assembler, or automate it ? With regard to manual conversions, I'm with you 100% - no thanks. :-)
But, my understanding is that the bulk of the work done in this case was done by a conversion tool that was designed and developed by a group of 8 people led by the gentleman referred to in the article (Jian Wang). The conversion tool project was the work that I was referring to in my comment.
Yes, manually. I missed that distinction, building the tool could be fun, but verifying it works would be a huge pain. Shouldn't be too hard to write test cases for tax code.
> Shouldn't be too hard to write test cases for tax code.
I would disagree here -- like timezones, tax software has the disadvantage of being both extremely boring, and very fiddly (thus requiring close attention). This combination makes it very hard to maintain the concentration that's required to write exhaustive tests.
Different strokes. Although, given a long enough period of working through assembly it might break the strongest of wills. I can imagine some people being very excited at setting up tests to ensure compatibility and watching them slowly going from fail to pass.
Rather than transcode assembly to a higher-level language. It's better to nail down all the functionality the assembly program provide and re-implement them in the higher-level language. Write language independent tests against the old code and re-run the tests against the new code to make sure things are working feature for feature and bug for bug.
This suggestion is pretty innocent of practical knowledge of the problem.
Simply extracting the control structure of the assembly via automation is a huge win. This type of assembly is incredibly dense - it was written for machines with at most, 100's of K of memory. It relies on highly non-local side effects which are very dependent on the data.
Things like a reference to a status register bit that was possibly set as a side effect of an instruction 30 or 40 instructions back, but then could have been modified by three or more other instructions, none of which are control flow instructions. All depending on knowing that certain patterns do/do not ever appear in the data.
After the control flow is extracted and made visible, then you can get to things like numerical emulation of the exact oddities of S360 code.
Once you have all those things done, which you will not ever, ever, be able to do by hand, then you can take the resulting generated Java and begin re-implementing.
The work being described has to happen before the suggestion above can even be approached.
There rarely is an original specification, and even if one exists, what was actually written has a high probably of not actually matching the specification exactly because of undocumented feature requests or additions.
Yeah, ~30 year projects were the longest i've ever worked on. Sometimes it's easier to read the docs about what the old programmers were trying to do - those telephone sized documents explaining requirements. if you're lucky, they might have started using source control in the 90's and you can kinda match up specs to commits.
It's the most difficult part of the project. However, nailing down the functionality upfront separates a successful project from a failed one. The implementation is the easy part. The functionality documentation will provide great value beyond this project.
I assume this assembly program does the tax calculation, so the IRS tax code can be consulted to verify the functionality. Functional tests can be written against the old code as the functionality is documented. The functional tests will guide the new implementation development, and serve as the acceptance tests of the new code.
You can even structure the process such that there's a team doing just the functional specification and functional test writing, and another team takes the result fed to them and does the implementation in parallel.
> It's the most difficult part of the project. However, nailing down the functionality upfront separates a successful project from a failed one.
Yes, in that projects which try to reimplement large existing systems from the ground up by nailing down the requirements and working from there, almost invariably fail (even if they formally “succeed” in the sense of being accepted and then facing years of remediation.)
You always want to do a Ship of Theseus replacement rather than all-at-once, if possible, and finding a way to lift-and-shift the existing implementation to a platform that supports the way you'd like to replace thing in the long term is a way of getting as close as possible to that when you can't practically do it directly.
This why java seems an odd choice; I can't imagine trying to run a codebase that's a split between assembly and java (keeping behaviour the same as more and more code is shifted).
The best idea I can see for that, is start with an interpreter for the assembler in java, moving procedure calls "up" into java.
But I imagine getting the interpreter good enough to run the same as the original target machine would be a huge task in itself.
[ed: leveraging clojure on the first iteration(s) might actually make the project feasible, though...]
I imagine you might get the interpreter running up to spec by flipping actual mainframe to debug, then writing a supervisor at the interpreter's side that checks for parity at each instruction, then single-steps the physical machine.
> the IRS tax code can be consulted to verify the functionality
You're not wrong, but the problem is often not that. The tax code helps when verifying the system end to end. But it's not like there's one single program that "does the tax" that you can verify that way.
The individual bits they're trying to replace are probably more like "fetch all these records from this one mainframe with format X and convert them this way, except if it's Feb 29th in which case fetch it from this other server and convert it another way and then pass it along, except for resident aliens which come from an entirely different place, fetch those from tape storage". This is much harder to get right without formal specs, which there almost certainly aren't.
In this case what we have is a "reference implementation" (a.k.a the implementation is the specification), and you can guess how well those go.
All this is not to say that they shouldn't have been doing what you're suggesting, but to say that you're making it sound easier than it is now that they're here.
That’s actually not true. I work for a fairly small contractor (a few hundred people) working on a fairly large government contract. It started with a handful of people when the company was a fraction of the size.
I'm very happy to be proven wrong. I must admit that I know very little about the process, other than what I've read, and what I've read hasn't been positive. But, I'm sure that small, successful projects aren't talked about nearly as much in the press as large, unsuccessful projects, so it could simply be bias in what I'm reading.
Lobbyists usually thwart any opportunity for someone cheap and legitimate to do the job. They've been trying to get someone to replace the humvee for decades and usually it's a 400 million dollar endeavor that results in a lesser product.
It's like these large companies take on the work, then instead of doing it they hire lawyers to figure out loopholes in the contract. Then they just pay lobbyists to get politicians to award them more contracts. All the while doing just enough to not get sued into oblivion.
All great ideas are spoiled by great bureaucrats and even greater lobbyists.
I've always thought that this type of government work should be open to bids from anyone. If the government is worried about trojans, malware, etc., then they can easily hire an auditor to audit the code and vouch for its authenticity. The fact that only very large, well-connected corporations get a crack at these types of problems is insane and a complete waste of taxpayer money.