Restructuring Git repo with prototype code to actual publishable project -

i've been lately working on api library wraps parts of relatively large external api more idiomatic structure. did api exploration while writing prototype code, ended implementing 3 of available sub-apis varying degrees of functionality. or put in simpler terms, have project structurally looks like

dir:root  └ dir:feature-a  └ dir:feature-b  └ dir:feature-c  └ dir:common  └ file:build.gradle  └ file:build.py 

where each feature matches 1 of sub-apis. worth mentioning directories aren't flat, omitted subdirectories simplicity's sake.

my main problem while did once provide semi decent version history, it's in 1 branch , 1 of sub-apis ready released. ideally, i'd find convenient way to

1. split existing repo can turn each feature own branch can publish them 1 one mature enough
2. keep current version history (with rebasing, possibly)

i have used git filter-branch similar purpose 1 major curve ball here repository root repository - on meta level repository has 2 parents admittedly funky , useful keeping build scripts up-to-date if tried want filter-branch build scripts @ root of project removed not want.

finally common directory bit special 1 - don't mind cutting version history, long contents there.

summary

if want retain history of common resources (build.*) , keep resources mergeable in future, and want rewrite/filter/remove sub-set of other trees in repository (feature-a, common) using git filter-branch, should first re-write existing commits in order:

• all common commits before project forked template (this case).
• all commits modifying build.*, including local changes , merges upstream cradle.
• finally, project-specific commits feature-* , common.

you can run git filter-branch safely on project-specific development-line, without rewriting of upstream resource history. if don't this, end-up re-writing commits involving build-scripts, including merge-commits upstream cradle, inhibit history traceability , future merges.

detail

it sounds have golden-project-template, call t, , each time start new project, fork repo (either in traditional github sense, or create divergent clone) call pn. pn , t start same history , common commits (call branch point pn-0).

as pn develops code-base, other projects might identify improvements base project-template infrastructure, , make change file f in t. project pn, might hundreds of commits ahead of template, can still merge-up changes in common files t.

now, want rewrite-history in pn. since pn-0 have made many project-specific commits, merge t, more project-specific commits. if had rewrite p pn-0 in order filter-branch, merge-history t lost, since histories have diverged, , future merges t become hellish.

does describe problem?

i think seeing using project-clone-from-template approach has limitations when want have full freedom of history-rewriting re-organise project repo. provided have history both before , after merge commits t, going have fancy re-organisation in order retain common history. solution is:

• let tx recent commit of t have performed full merge of pn.
• fetch t pn repo, , create branch in pn starts commit tx.
• rebase current pn history onto branch, moving base of pn-0 (common commit t) tx, latest common commit t.

this approach replay entire history in pn if started tx instead of pn-0, commit pn-1 has new parent tx. of course each commit re-written, existing clones of pn orphaned.

once have this, free run git filter-branch starting re-written commit pn-1, , remove history of incomplete modules.

now - fair amount of trouble go to, , rewrites history in tricky ways, history retained. wouldn't want doing process every day.

one thing might want consider whether there's way can produce , consume cradle without source-sharing. might not convenient git-merging, if template project version-controlled , organize build logic , maybe use shared scripts, can modularize template project no longer depend on child-projects maintaining common source histories in order merge-up - consume latest template binaries instead. depends lot of course on what's in template other build logic.