How can I fetch git objects using the smart protocol (raw) over http? -


i'm trying fetch annotation of tag "v2.4.2" github.com/git/git using git smart protocol on http.

// refs

curl -h "user-agent: git/1.8.1" -v  https://github.com/git/git/info/refs?service=git-upload-pack

returns refs:

..... 003e2be062dfcfd1fd4aca132ec02a40b56f63776202 refs/tags/v2.4.1 0041aaa7e0d7f8f003c0c8ab34f959083f6d191d44ca refs/tags/v2.4.1^{} 003e29932f3915935d773dc8d52c292cadd81c81071d refs/tags/v2.4.2 00419eabf5b536662000f79978c4d1b6e4eff5c8d785 refs/tags/v2.4.2^{}

// make upload pack request

printf "0031want 00419eabf5b536662000f79978c4d1b6e4eff5c8d785\n0024have 003e2be062dfcfd1fd4aca132ec02a40b56f63776202\n0000" | curl -h "user-agent: git/1.8.1" -v  -d @- https://github.com/git/git/git-upload-pack -h "content-type: application/x-git-upload-pack-request" --trace-ascii /dev/stdout

this returns nothing. i'm wondering what's wrong in request (i.e did miscalculate hex?)

warning: --trace-ascii overrides earlier trace/verbose option == info: hostname not found in dns cache == info:   trying 192.30.252.130... == info: connected github.com (192.30.252.130) port 443 (#0) == info: tls 1.2 connection using tls_ecdhe_rsa_with_aes_128_cbc_sha256 == info: server certificate: github.com == info: server certificate: digicert sha2 extended validation server ca == info: server certificate: digicert high assurance ev root ca => send header, 170 bytes (0xaa) 0000: post /git/git/git-upload-pack http/1.1 0028: host: github.com 003a: accept: */* 0047: user-agent: git/1.8.1 005e: content-type: application/x-git-upload-pack-request 0093: content-length: 110 00a8:  => send data, 110 bytes (0x6e) 0000: 0031want 00419eabf5b536662000f79978c4d1b6e4eff5c8d7850024have 00 0040: 3e2be062dfcfd1fd4aca132ec02a40b56f637762020000 == info: upload sent off: 110 out of 110 bytes <= recv header, 17 bytes (0x11) 0000: http/1.1 200 ok == info: server github babel 2.0 not blacklisted <= recv header, 26 bytes (0x1a) 0000: server: github babel 2.0 <= recv header, 52 bytes (0x34) 0000: content-type: application/x-git-upload-pack-result <= recv header, 28 bytes (0x1c) 0000: transfer-encoding: chunked <= recv header, 40 bytes (0x28) 0000: expires: fri, 01 jan 1980 00:00:00 gmt <= recv header, 18 bytes (0x12) 0000: pragma: no-cache <= recv header, 53 bytes (0x35) 0000: cache-control: no-cache, max-age=0, must-revalidate <= recv header, 23 bytes (0x17) 0000: vary: accept-encoding <= recv header, 2 bytes (0x2) 0000:  <= recv data, 5 bytes (0x5) 0000: 0 0003:  == info: connection #0 host github.com left intact

why trying this?

  • i don't have write access file system
  • avoid fetching unnecessary data (i.e commits)
  • standard api/protocol

commit hex

you didn't miscalculate hex, you're not passing correct value. remember each line in smart protocol preceded length count:

<length><data>

so line looks this:

00419eabf5b536662000f79978c4d1b6e4eff5c8d785 refs/tags/v2.4.2^{}

you need discard first 4 characters, makes actual commit hex:

9eabf5b536662000f79978c4d1b6e4eff5c8d785

request format

when posting request, have , want lines supposed separated newline, if take @ output curl, can see there no newline:

=> send data, 110 bytes (0x6e) 0000: 0031want 00419eabf5b536662000f79978c4d1b6e4eff5c8d7850024have 00 0040: 3e2be062dfcfd1fd4aca132ec02a40b56f637762020000

you need use --data-binary instead of --data:

--data-binary @-

you need prefix these lines length count, , need end line consisting of 0000:

0032want 9eabf5b536662000f79978c4d1b6e4eff5c8d785 0032have 2be062dfcfd1fd4aca132ec02a40b56f63776202 0000

debugging tips

you can set git_trace_packet=1 in environment if want copious debugging information git see it's sending , forth.

and that's wrote

i'm not able response myself, given above information, figured help.

update

so, fun.

i set git server locally (using git http-backend , thttpd), , ran tcpdump grab traffic generated git remote update operation. turns out need separate want , have directives null command, 0000 (no newline, because length encodes newlines, too). is:

<length>want <commitid><newline> 0000<length>have <commitid><newline> <length>done

e.g:

0032want 9eabf5b536662000f79978c4d1b6e4eff5c8d785 00000032have 2be062dfcfd1fd4aca132ec02a40b56f63776202 0009done

that gives me:

0000: post /git/git/git-upload-pack http/1.1 0028: host: github.com 003a: accept: */* 0047: content-type: application/x-git-upload-pack-request 007c: user-agent: git/1.8 0091: content-length: 113 00a6:  => send data, 113 bytes (0x71) 0000: 0032want 9eabf5b536662000f79978c4d1b6e4eff5c8d785.00000032have 2 0040: be062dfcfd1fd4aca132ec02a40b56f63776202.0009done. == info: upload sent off: 113 out of 113 bytes <= recv header, 17 bytes (0x11) 0000: http/1.1 200 ok <= recv header, 26 bytes (0x1a) 0000: server: github babel 2.0 <= recv header, 52 bytes (0x34) 0000: content-type: application/x-git-upload-pack-result <= recv header, 28 bytes (0x1c) 0000: transfer-encoding: chunked <= recv header, 40 bytes (0x28) 0000: expires: fri, 01 jan 1980 00:00:00 gmt <= recv header, 18 bytes (0x12) 0000: pragma: no-cache <= recv header, 53 bytes (0x35) 0000: cache-control: no-cache, max-age=0, must-revalidate <= recv header, 23 bytes (0x17) 0000: vary: accept-encoding <= recv header, 2 bytes (0x2) 0000:  <= recv data, 4 bytes (0x4) 0000: 31 <= recv data, 51 bytes (0x33) 0000: 0031ack 2be062dfcfd1fd4aca132ec02a40b56f63776202. <= recv data, 6 bytes (0x6) 0000: 1fff <= recv data, 1370 bytes (0x55a) 0000: pack.......[..x...an.0...z.?`..d.*..@..z..(.tu......>~b.....]..8 0040: 2...j).oq}..#.....'......[..8k..t..,%.s..u..@l..xt...o......'... [....]

double-bonus update

you can use git unpack-objects command extract packfile. can see above trace, first length-encoded response ( 0031ack 2be062dfcfd1fd4aca132ec02a40b56f63776202) followed pack data, need discard first line:

$ git init tmprepo $ cd temprepo $ tail -n +2 output_from_curl | git unpack-objects unpacking objects: 100% (91/91), done. $ find .git/objects -type f | head -3 $ git cat-file -p dc940e63c453199dd9a7285533fbf2355bab03d1 /*  * git - information manager hell  *  * copyright (c) linus torvalds, 2005  *  * handles basic git sha1 object files - packing, unpacking,  * creation etc.  */ [...]

Comments

Post a Comment

Popular posts from this blog

angularjs - ADAL JS Angular- WebAPI add a new role claim to the token -

i have created angular spa webapi backend using adal js authentication.as there no roles in ad, need manually add role claims in order give users access different api controllers. the roles stored in database. expecting inject claim through call webapi after authentication ad. webapi code might this. identity.addclaim(new claim("role", "user")); var ticket = new authenticationticket(identity, props); var accesstoken = startup.oauthbeareroptions.accesstokenformat.protect(ticket); is possible replace adal idtoken new token? is viable solution or there other better way handle this? as initial token generated azuread, possible edit token add new claim? appreciated. graph api supporting group claims. see here: http://justazure.com/azure-active-directory-part-4-group-claims/ if @ examples on page, users assigned groups , app can check group in claims. in current version of portal, need app manifest , modify it.
Read more

node.js - Using Node without global install -

i'm working on few node & electron (atom shell) projects , i'm curious how them work without user having install node globally. i've seen few applications including node directory in project contains node.exe binary. how go setting node project use binary? edit: thought should add, i've searched everywhere answer this, unrelated results.
Read more

php - CakePHP HttpSockets send array of paramms -

i problem when make request (i can request). $link = 'http://uri/path/?param1=1&sid=2&sid=3&sid=4' $http = $httpsocket = new httpsocket(); $res = $httpsocket->get($link); if check request pr($http); i see : [line] => /url/path/?param1=1&sid%5b0%5d=2&sid%5b1%5d=3&sid%5b2%5d=4 http/1.1 and i'm getting empty response because server doesn't know parameters sid%5b0%5d=2 when try send parameters array : $data = array('param1' => '1', 'sid' => array('2','3', '4')); i see same changes %5b0%5d - additional indexes. how fix it? unfortunately , can not change server side, if send request in browser, i'll normal response in json format. when make request, can parameters this: $param1 = $_get['param1']; $sid= $_get['sid']; tip: use diferente names parameters. have multiples sid. use sid1, sid2, sid3... update: can parameters this:...
Read more