1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
|
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="stylesheet" href="css/simple.css">
<link rel="stylesheet" href="css/screen.css">
<title>Building a client - Documentation</title>
</head>
<body>
<header>
<h1>Documentation</h1>
<p>eduVPN for Linux</p>
<nav>
<a href="index.html">About</a>
<a href="building.html">Building</a>
<a href="testing.html">Testing</a>
<a href="building-client.html">Building a client</a>
<a href="apidocs.html">API Docs</a>
</nav>
</header>
<main>
<aside>
<ul>
<li class="nav-item" data-level="2"><a href="#go-language-x-interop" class="nav-link">Go <-> language X interop</a>
<ul class="nav flex-column">
</ul>
</li>
<li class="nav-item" data-level="2"><a href="#architecture" class="nav-link">Architecture</a>
<ul class="nav flex-column">
</ul>
</li>
<li class="nav-item" data-level="2"><a href="#typical-flow-for-a-client" class="nav-link">Typical flow for a client</a>
<ul class="nav flex-column">
</ul>
</li>
<li class="nav-item" data-level="2"><a href="#finite-state-machine" class="nav-link">Finite state machine</a>
<ul class="nav flex-column">
<li class="nav-item" data-level="3"><a href="#fsm-example" class="nav-link">FSM example</a>
<ul class="nav flex-column">
</ul>
</li>
<li class="nav-item" data-level="3"><a href="#state-explanation" class="nav-link">State explanation</a>
<ul class="nav flex-column">
</ul>
</li>
<li class="nav-item" data-level="3"><a href="#states-that-ask-data" class="nav-link">States that ask data</a>
<ul class="nav flex-column">
</ul>
</li>
</ul>
</li>
<li class="nav-item" data-level="2"><a href="#code-examples" class="nav-link">Code examples</a>
<ul class="nav flex-column">
<li class="nav-item" data-level="3"><a href="#go-command-line-client" class="nav-link">Go command line client</a>
<ul class="nav flex-column">
</ul>
</li>
</ul>
</li>
</ul>
</aside>
<h1 id="building-a-client">Building a client<a class="headerlink" href="#building-a-client" title="Permanent link">#</a></h1>
<p>This chapter is a high-level overview on how to use eduvpn-common and build your own eduVPN/Let’s Connect! client. In this chapter, we go over the basics of how the interop between Go and language x works, say something about the architecture, explain where to find detailed API documentation, explain the state machine, give a typical flow for a client and give a follow along tutorial on building an eduVPN client using Python code. At last, we will also have a few code examples that can be used as a short reference.</p>
<h2 id="go-language-x-interop">Go <-> language X interop<a class="headerlink" href="#go-language-x-interop" title="Permanent link">#</a></h2>
<p>Because this library is meant to be a <em>general</em> library for other clients to use that are written in different programming languages, we need to find a way to make this Go library available on each platform and codebase. The approach that we take is to build a C library from the Go library using Cgo. Cgo can have its disadvantages with performance and the constant conversion between Go and C types. To overcome those barriers, this library has the following goals (with some others noted here):</p>
<ul>
<li><strong>Be high-level</strong>. Functions should do as much as possible in Go. The exported API should fit in one file. Lots of low-level functions would be a constant conversion between C and Go which adds overhead</li>
<li><strong>Move as much state to Go as possible</strong>. For example, Go keeps track of the servers you have configured and discovery. This makes the arguments to functions simple, clients should pass simple identifiers that Go can look up in the state</li>
<li><strong>Easy type conversion</strong>: to convert between C and Go types, JSON is used. Whereas Protobuf, Cap’n’proto or flatbuffers are more performant, they are harder to debug, add thousands of lines of autogenerated code and are not human friendly. Using JSON, the clients can approach it the same way they would use with a server using a REST API. Another approach is to just convert from Go -> C types -> language types. This was tried in version 1 of the library, but this ended up being too much work and manual memory management</li>
<li><strong>Make it as easy as possible for clients to manage UI and internal state</strong>: we use a state machine that gives the clients information in which state the Go library is in, e.g. we’re selecting a server profile, we’re loading the server endpoints. This library is not only a layer to talk to eduVPN servers, but the whole engine for a client</li>
<li><strong>Implement features currently not present in existing clients</strong>: WireGuard to OpenVPN failover, WireGuard over TCP</li>
<li><strong>Follow the official eduVPN specification</strong> and also contribute changes when needed</li>
<li><strong>Secure</strong>: We aim to follow the latest OAuth recommendations, to not store secret data and e.g. disable OpenVPN scripts from being ran by default</li>
</ul>
<p>And finally the most important goal:</p>
<ul>
<li><strong>The advantages that this library brings for clients should outweigh the cost of incorporating it into the codebase</strong>. Initial versions would take more work than we get out of it. However, when each eduVPN/Let’s Connect! client uses this library we should expect a net gain. New features should be easier to implement for clients by simply requiring a new eduvpn-common version and using the necessary functions</li>
</ul>
<h2 id="architecture">Architecture<a class="headerlink" href="#architecture" title="Permanent link">#</a></h2>
<p>In the previous section, we have already hinted a bit on the exact architecture. This section will expand upon it by giving a figure of the basic structure</p>
<p><img alt="overview of shared library structure" src="overview.mmd.svg" /></p>
<p>As can be seen by this architecture, there is an intermediate layer between the client and the <em>shared</em> library. This wrapper eases the way of loading this library and then defining a more language specific API for it. In the eduvpn-common repo, we currently only support a Python wrapper. Clients themselves can define their own wrapper</p>
<h2 id="typical-flow-for-a-client">Typical flow for a client<a class="headerlink" href="#typical-flow-for-a-client" title="Permanent link">#</a></h2>
<blockquote>
<p><strong><em>NOTE:</em></strong> This uses the function names that are defined in the exports file in Go. For your own wrapper/the Python wrapper they are different. But the general flow is the same</p>
</blockquote>
<ol>
<li>The client starts up. It calls the <code>Register</code> function that communicates with the library that it has initialized</li>
<li>It gets the list of servers using <code>ServerList</code></li>
<li>
<p>When the user selects a server to connect to in the UI, it calls the <code>GetConfig</code> to get a VPN configuration for this server. This function transitions the state machine multiple times. The client uses these state transitions for logging or even updating the UI. The client then connects</p>
<ul>
<li>New feature in eduvpn-common: Check if the VPN can reach the gateway after the client is connected by calling <code>StartFailover</code></li>
</ul>
</li>
<li>
<p>If the client has no servers, or it wants to add a new server, the client calls <code>DiscoOrganizations</code> and <code>DiscoServers</code> to get the discovery files from the library. This even returns cached copies if the organizations or servers should not have been updated <a href="https://docs.eduvpn.org/server/v3/server-discovery.html">according to the documentation</a></p>
<ul>
<li>From this discovery list, it calls <code>AddServer</code> to add the server to the internal server list of eduvpn-common. This also calls necessary state transitions, e.g. for authorizing the server. The next call to <code>ServerList</code> then has this server included</li>
<li>It can then get a configuration for this server like we have explained in <em>step 3</em></li>
</ul>
</li>
<li>
<p>When a configuration has been obtained, the internal state has changed and the client can get the current server that was configured using <code>CurrentServer</code>. <code>CurrentServer</code> can also be called after startup if a server was previously set as the current server</p>
</li>
<li>When the VPN disconnects, the client calls <code>Cleanup</code> so that the server resources are cleaned up by calling the <code>/disconnect</code> endpoint</li>
<li>A server can be removed with the <code>RemoveServer</code> function</li>
<li>When the client is done, it calls <code>Deregister</code> such that the most up to date internal state is saved to disk. Note that eduvpn-common also saves the internal state .e.g. after obtaining a VPN configuration</li>
</ol>
<h2 id="finite-state-machine">Finite state machine<a class="headerlink" href="#finite-state-machine" title="Permanent link">#</a></h2>
<p>The eduvpn-common library uses a finite state machine internally to keep track of which state the client is in and to communicate data callbacks (e.g. to communicate the Authorization URL in the OAuth process to the client).</p>
<h3 id="fsm-example">FSM example<a class="headerlink" href="#fsm-example" title="Permanent link">#</a></h3>
<p>The following is an example of the FSM when the client has obtained a Wireguard/OpenVPN configuration from an eduVPN server</p>
<p><img alt="finite state machine (FSM) of eduvpn-common" src="fsm.mmd.svg" /></p>
<p>The current state is highlighted in the <span style="color:cyan">cyan</span> color.</p>
<h3 id="state-explanation">State explanation<a class="headerlink" href="#state-explanation" title="Permanent link">#</a></h3>
<p>For the explanation of what all the different states mean, see the <a href="https://codeberg.org/eduVPN/eduvpn-common/src/branch/main/client/fsm.go#L22-L58">client documentation</a></p>
<h3 id="states-that-ask-data">States that ask data<a class="headerlink" href="#states-that-ask-data" title="Permanent link">#</a></h3>
<p>In eduvpn-common, there are certain states that require attention from the client.</p>
<ul>
<li>OAuth Started: A state that must be handled by the client. How a client can ‘handle’ this state, we will see in the next section. In this state, the client must open the webbrowser with the authorization URL to complete to OAuth process. Note that on mobile platforms, you also need to reply with the authorization URI as these platforms do not support a local callback server using 127.0.0.1</li>
<li>Ask Profile: The state that asks for a profile selection to the client. Reply to this state by using a “cookie” and the CookieReply function. What this means will be discussed in the Python client example too</li>
<li>Ask Location: Same for ask profile but for selecting a secure internet location. Only called if one must be chosen, e.g. due to a selection that is no longer valid</li>
</ul>
<p>The rest of the states are miscellaneous states, meaning that the client can handle them however it wants to. However, it can be useful to handle most state transitions to e.g. show loading screens or for logging and debugging purposes.</p>
<h2 id="code-examples">Code examples<a class="headerlink" href="#code-examples" title="Permanent link">#</a></h2>
<p>This chapter contains code examples that use the API</p>
<h3 id="go-command-line-client">Go command line client<a class="headerlink" href="#go-command-line-client" title="Permanent link">#</a></h3>
<p>The following is an example <a href="https://codeberg.org/eduVPN/eduvpn-common/src/branch/main/cmd/eduvpn-cli/main.go">in the repository</a>. It is a command line client.</p>
<pre><code class="language-go">{// Package main implements an example CLI client
package main
import (
"context"
"flag"
"fmt"
"os"
"reflect"
"strings"
"codeberg.org/eduVPN/eduvpn-common/client"
"codeberg.org/eduVPN/eduvpn-common/i18n"
"codeberg.org/eduVPN/eduvpn-common/internal/commonver"
"codeberg.org/eduVPN/eduvpn-common/types/cookie"
srvtypes "codeberg.org/eduVPN/eduvpn-common/types/server"
"github.com/pkg/browser"
)
// Open a browser with xdg-open.
func openBrowser(data any) {
str, ok := data.(string)
if !ok {
return
}
go func() {
err := browser.OpenURL(str)
if err != nil {
fmt.Fprintln(os.Stderr, "failed to open browser with error:", err)
fmt.Println("Please open your browser manually")
}
}()
}
func getProfileInteractive(profiles *srvtypes.Profiles, data any) (string, error) {
fmt.Printf("Multiple VPN profiles found. Please select a profile by entering e.g. 1")
var ps strings.Builder
var options []string
i := 0
for k, v := range profiles.Map {
fmt.Fprintf(&ps, "\n%d - %s", i+1, i18n.GetLanguageMatched(v.DisplayName, "en"))
options = append(options, k)
i++
}
// Show the profiles
fmt.Println(ps.String())
var idx int
if _, err := fmt.Scanf("%d", &idx); err != nil || idx <= 0 ||
idx > len(profiles.Map) {
fmt.Fprintln(os.Stderr, "invalid profile chosen, please retry")
return getProfileInteractive(profiles, data)
}
p := options[idx-1]
fmt.Println("Sending profile ID", p)
return p, nil
}
func sendProfile(profile string, data any) {
d, ok := data.(*srvtypes.RequiredAskTransition)
if !ok {
fmt.Fprintf(os.Stderr, "\ninvalid data type: %v\n", reflect.TypeOf(data))
os.Exit(1)
}
sps, ok := d.Data.(*srvtypes.Profiles)
if !ok {
fmt.Fprintf(os.Stderr, "\ninvalid data type for profiles: %v\n", reflect.TypeOf(d.Data))
os.Exit(1)
}
if profile == "" {
gprof, err := getProfileInteractive(sps, data)
if err != nil {
fmt.Fprintf(os.Stderr, "failed getting profile interactively: %v\n", err)
os.Exit(1)
}
profile = gprof
}
if err := d.C.Send(profile); err != nil {
fmt.Fprintf(os.Stderr, "failed setting profile with error: %v\n", err)
os.Exit(1)
}
}
// The callback function
// If OAuth is started we open the browser with the Auth URL
// If we ask for a profile, we send the profile using command line input
// Note that this has an additional argument, the vpn state which was wrapped into this callback function below.
func stateCallback(_ client.FSMStateID, newState client.FSMStateID, data any, prof string, dir string) {
if newState == client.StateOAuthStarted {
openBrowser(data)
}
if newState == client.StateAskProfile {
sendProfile(prof, data)
}
if newState == client.StateAskLocation {
// removing is best effort
_ = os.RemoveAll(dir)
fmt.Fprint(os.Stderr, "An invalid secure location is stored. This CLI doesn't support interactively choosing a location yet. Give a correct location with the -country-code flag")
os.Exit(1)
}
}
// Get a config for Institute Access or Secure Internet Server.
func getConfig(state *client.Client, url string, srvType srvtypes.Type, cc string, prof string) (*srvtypes.Configuration, error) {
if !strings.HasPrefix(url, "http") {
url = "https://" + url
}
ck := cookie.NewWithContext(context.Background())
defer ck.Cancel() //nolint:errcheck
err := state.AddServer(ck, url, srvType, nil)
if err != nil {
// TODO: This is quite hacky :^)
if !strings.Contains(err.Error(), "a secure internet server already exists.") {
return nil, err
}
}
if cc != "" {
err = state.SetSecureLocation(url, cc)
if err != nil {
return nil, err
}
}
if prof != "" {
// this is best effort, e.g. if no server was chosen before this fails
_ = state.SetProfileID(prof) //nolint:errcheck
}
return state.GetConfig(ck, url, srvType, false, false)
}
// Get a config for a single server, Institute Access or Secure Internet.
func printConfig(url string, cc string, srvType srvtypes.Type, prof string) error {
var c *client.Client
var err error
var dir string
dir, err = os.MkdirTemp("", "eduvpn-common")
if err != nil {
return err
}
// removing is best effort
defer os.RemoveAll(dir) //nolint:errcheck
c, err = client.New(
"org.eduvpn.app.linux",
fmt.Sprintf("%s-cli", commonver.Version),
dir,
func(oldState client.FSMStateID, newState client.FSMStateID, data any) bool {
stateCallback(oldState, newState, data, prof, dir)
return true
},
nil,
)
if err != nil {
return err
}
_ = c.Register()
ck := cookie.NewWithContext(context.Background())
_, err = c.DiscoOrganizations(ck, false, "")
if err != nil {
return err
}
_, err = c.DiscoServers(ck, false, "")
if err != nil {
return err
}
defer c.Deregister()
cfg, err := getConfig(c, url, srvType, cc, prof)
if err != nil {
return err
}
fmt.Println(cfg.VPNConfig)
return nil
}
// The main function
// It parses the arguments and executes the correct functions.
func main() {
cu := flag.String("get-custom", "", "The url of a custom server to connect to")
u := flag.String("get-institute", "", "The url of an institute to connect to")
sec := flag.String("get-secure", "", "Gets secure internet servers")
cc := flag.String("country-code", "", "The country code to use in case of a secure internet server")
prof := flag.String("profile", "", "The profile ID to choose")
flag.Parse()
// Connect to a VPN by getting an Institute Access config
var err error
switch {
case *cu != "":
err = printConfig(*cu, "", srvtypes.TypeCustom, *prof)
case *u != "":
err = printConfig(*u, "", srvtypes.TypeInstituteAccess, *prof)
case *sec != "":
err = printConfig(*sec, *cc, srvtypes.TypeSecureInternet, *prof)
default:
flag.PrintDefaults()
}
if err != nil {
fmt.Fprintf(os.Stderr, "failed to get a VPN config: %v\n", err)
}
}}
</code></pre>
</main>
<footer>
<p>Documentation - eduVPN for Linux</p>
</footer>
</body>
</html>
|
